Warn when SKILL.md contains invisible characters - #6098
Warn when SKILL.md contains invisible characters#6098fevziegeyurtsevenler wants to merge 1 commit into
Conversation
collectWarnings already receives the raw SKILL.md bytes but only looks at formatting: comma-delimited allowed-tools and line count. Characters that render as nothing still reach the model as instruction text, so a skill can carry directions a reviewer reading the rendered file cannot see. Flag Unicode tag characters (U+E0000-E007F, a full invisible ASCII alphabet and the usual carrier), zero-width characters, invisible math operators, bidirectional overrides of Trojan Source fame, and variation selectors, naming the kind found and the first line it appears on. This is a warning rather than an error on purpose: ZWJ appears in ordinary emoji sequences and RTL marks appear in ordinary Arabic and Hebrew prose, so the call belongs to a human. Ordinary non-ASCII prose stays clean, which the tests cover alongside each flagged class. Signed-off-by: Fevzi Ege Yurtsevenler <egeyurtsevenler@gmail.com>
|
Hi @fevziegeyurtsevenler — thank you for this, genuinely. It's a thoughtful, well-researched contribution: the Unicode tag-block and Trojan Source (CVE-2021-42574) cases are exactly the kind of thing we'd want skill validation to catch, and the writeup, the warning-not-error reasoning, the test coverage, and the DCO sign-off are all in great shape. Before we take a change into review, though, we do ask that it go through our contribution process. Per the CONTRIBUTING guide, the first step is to open an issue outlining the fix or feature so we can align on scope and design before code is written. For something in security-sensitive validation like this, that up-front discussion is especially valuable — it lets us agree on which codepoint classes to flag, the warning-vs-error semantics, and how it fits alongside the rest of skill validation, so your effort lands cleanly instead of getting reworked in review. So I'm going to close this PR for now — please don't read that as a rejection of the idea, which we like. When you have a moment, could you:
Thanks again for the care you put into this — we'd be glad to see it come back through that route. 🙏 |
Adds an invisible-character warning to skill validation.
Problem
collectWarningsinpkg/skills/validator.goalready receives the rawSKILL.mdbytes, but only inspects formatting — comma-delimitedallowed-toolsand line count. Nothing looks at the characters themselves.Characters that render as nothing still reach the model as instruction text. A skill can therefore carry directions that a reviewer reading the rendered file cannot see. The clearest case is the Unicode tag block (U+E0000–E007F), a complete ASCII alphabet that essentially every renderer draws as zero pixels:
Bidirectional overrides are the other well-known case — the rendered order of a line can differ from the byte order that gets read, which is Trojan Source / CVE-2021-42574.
Change
checkInvisibleRunesscans the content once and reports the classes found plus the first line they appear on:Why a warning and not an error
These codepoints have legitimate uses — ZWJ builds ordinary emoji sequences, and RTL marks appear throughout ordinary Arabic and Hebrew prose. Failing validation on them would break real skills, so this surfaces the finding and leaves the judgement to a human, consistent with how the existing warnings behave.
Ordinary non-ASCII prose stays clean; there is a test asserting that Turkish, Greek, Japanese and emoji content produces no warning, alongside one test per flagged class and one for the reported line number.
Verification
go test ./pkg/skills/... -count=1,go vet ./pkg/skills/andgofmt -l pkg/skills/are all clean locally. Commit is DCO signed-off.Context: I maintain uncloak, a scanner for this class of issue, and ran it across 3,168 public agent extensions — which is where the specific codepoint classes above come from.