feat(input): OSC22#101
Conversation
Terminals implementing the kitty mouse-pointer-shape protocol answer an
OSC 22 query on the input stream. The parser now recognizes these replies
(ESC ] 22 ; <payload> ST|BEL) and surfaces them as a new InputEvent
variant carrying the raw payload string.
The payload is surfaced verbatim, not interpreted: the same reply shape
covers a current-shape name, "0" for an empty stack, and a support-query
list ("1,0,1"). Which applies depends on the query the caller sent, so
correlation is the caller's responsibility — keeping the parser stateless
and independent of the renderer (INV-8). Emitting the queries and setting
shapes is an output concern and lives elsewhere.
Set-only terminals (e.g. Ghostty) never reply, so they never produce the
event; absence within a timeout is the unsupported contract, not an error.
Lets the mouse pointer change shape over the UI, driven by the renderer's existing hit-testing. Elements declare a CSS-style `cursor` shape on open() (renderer-ignored, not packed to wasm); with `trackCursor: true`, render() returns OSC 22 bytes in a separate `cursor` field for the caller to write — kept out of `output` so render content stays pure. Narrows the §11.2 prohibition to the text caret and carves out a single opt-in exception for the mouse pointer shape, preserving INV-1 (renderer produces bytes, caller writes) and INV-7 (capability replies arrive via the input-side PointerShapeEvent; the caller correlates them). All tracking state lives in the TS term layer, so the wasm core stays frame-stateless. Framed as elastic §12 surface, like the pointer event model it builds on.
Elements declare a CSS-style `cursor` shape on open(); it is a pure annotation, never packed to the WASM module. With `trackCursor: true`, render() finds the topmost element under the pointer that declares a cursor and returns the OSC 22 bytes for any change in `result.cursor`, kept separate from `output` so render content stays pure (§11.2). Save/restore uses the kitty pointer-shape stack: enter pushes, leave pops, so the terminal's prior shape is restored without a query. All tracking state lives in the TS term layer alongside the existing pointer-enter/leave bookkeeping; the wasm core is untouched. Pointer-over ids are outermost-first, so topmost-wins scans from the end. Adds set/push/pop/query OSC 22 byte helpers and a CursorShape (CSS cursor keyword) type to termcodes.
Declares cursor: "pointer" on each key and enables trackCursor, writing the returned OSC 22 bytes so the mouse pointer turns into a hand over the keys and reverts elsewhere.
The tracker emitted the kitty push/pop stack form (ESC]22;>shape and ESC]22;<). Ghostty's OSC 22 parser treats the whole payload after "22;" as a literal shape name, so ">shape" is not a valid shape and is dropped — the pointer never changed on Ghostty (and any other set-only terminal). Switch to the portable bare set form: set the shape on enter (ESC]22;shape) and restore the base by setting "default" on leave. kitty and Ghostty both honor this. The trade-off is that we assume the base shape is "default" rather than restoring a non-default prior shape; the push/pop helpers remain exported for callers that target kitty and want exact save/restore.
|
Size Increased — +8.4 KB 121.0 KB unpacked |
commit: |
dreyfus92
left a comment
dreyfus92
left a comment
There was a problem hiding this comment.
awesome job nate 😁 left a few comments are minimal, just wanna make sure they're intended by design or something wasn't considered while working on this 👀
| int code = -1; | ||
| while (i < st->len && st->buf[i] >= '0' && st->buf[i] <= '9') { | ||
| if (code == -1) | ||
| code = 0; | ||
| code = code * 10 + (st->buf[i] - '0'); | ||
| i++; |
There was a problem hiding this comment.
the loop accumulates digits with no cap before the code != 22. feed it ESC ] 99999999999... and code overflows int. simplest guard is to clamp or bail inside the loop, e.g. break out once code > 22 (or some small ceilling) since the only code you accept is 22 anyway anything is already a PARSE_ERR.
| let cursor: Uint8Array | undefined; | ||
| if (options?.trackCursor) { | ||
| // Set-only OSC 22: the base is "default" (kitty and Ghostty both honor | ||
| // a bare set; the kitty push/pop stack is ignored by set-only terminals | ||
| // like Ghostty). | ||
| let active: CursorShape = "default"; | ||
| if (overIds.length > 0) { | ||
| let shapes = new Map<string, CursorShape>(); | ||
| for (let op of ops) { | ||
| if (isOpen(op) && op.cursor) shapes.set(op.id, op.cursor); | ||
| } | ||
| // pointerOverIds is outermost-first; the innermost (topmost) | ||
| // declaring element wins, so scan from the end. | ||
| for (let i = overIds.length - 1; i >= 0; i--) { | ||
| let shape = shapes.get(overIds[i]); | ||
| if (shape) { | ||
| active = shape; | ||
| break; | ||
| } | ||
| } | ||
| } | ||
| if (active !== cursorShape) { | ||
| cursor = POINTERSHAPE(active); | ||
| cursorShape = active; | ||
| } | ||
| } | ||
|
|
There was a problem hiding this comment.
the reset only happens inside this block, so if a caller stops passing trackCursor while cursorShape is non def, nothing emits POINTERSHAPE("default") and the terminal keeps the custom pointer. same on exit, set-only can't self restore, so a non def shape just lingers.
could track whether trackCursor was on last frame and emit one POINTERSHAPE("default") on the on off flip. the exit case probably just needs a doc note that the caller resets before teardown. fine as a follow-up if you want to keep this scoped.
cowboyd
left a comment
cowboyd
left a comment
There was a problem hiding this comment.
Keeping the pointer edits out of band is definitely safe, but after having a sit with it, I'm wondering what the downside would be of just making the pointer edits part of the main output? In other words, if you declare up front in createTerm() that you want to emit pointer glyph edits, then it keeps the write simple. process.stdout.writeSync(result.output)
You'll still have to do the work to decide whether your terminal emulator supports it, but if you front load that complexity, then it cuts down considerably on the complexity of each paint.
We could also add some sort of probe() function which lets you pass in terminfo and it will emit a query and parse results to see what is supported for a session. If we had that, then we could even add an autoprobe? option to createTerm()
Open Questions
- Do we support pointer edits when rendering in line mode? probably not? But worth asking.
3 participants
What does this PR do?
Adds OSC22 support (pointerShape). Because the specs dictate that renderer/input must be separated (INV-7/INV-8), this is surfaced by the host loop writing
result.cursorto stdin.Clay_GetPointerOverIds()against a new element-levelcursorprop and emitsOSC 22set bytes in a newresult.cursor. There is no write, this is a pure annotation.parse_osc()decodes the terminal's reply to a shape query (ESC]22;<payload>ST|BEL) into aPointerShapeEvent{ report }If there's another architecture or a more generic way to handle OSC sequences (like a dedicated OSC event), I am totally up for adjusting the approach.
Type of change
Checklist
pnpm test)pnpm format)AI-generated code disclosure