Skip to content

refactor: migrate focus to the request-bound device runtime - #1925

Merged
thymikee merged 3 commits into
mainfrom
agent/wave5-focus
Aug 21, 2026
Merged

refactor: migrate focus to the request-bound device runtime#1925
thymikee merged 3 commits into
mainfrom
agent/wave5-focus

Conversation

@thymikee

@thymikee thymikee commented Aug 20, 2026

Copy link
Copy Markdown
Member

Wave 5's first unit for #1739 (ADR 0019). Migrates focus from the legacy interactor leaf onto a
request-bound device runtime.

Unit record

  • Denominator cells: one descriptor (focus), one operation (focusPoint), six platform
    families plus two provider runtimes.
  • Parity source: handleFocusCommand in src/core/dispatch-interactions.ts and the focus
    arm of the dispatchKnownCommand table, at 06d27de4d.
  • Facet owner: packages/contracts/src/focus-runtime.ts — one contract owner for the
    operation, its facts, and both interactor binders.
  • Evidence tier: request-scoped. focus binds no durable resource; its binding lives exactly
    as long as the request.
  • Expected deletions: handleFocusCommand, the dispatch-table arm, the focus capability
    bucket, and focus's membership in the two hand-maintained capability overlays.
  • Size budget: move-dominated, no net production growth beyond the new contract. Actual below.

What changed

One execution path. focus x y (generic route) and find <q> focus both resolve
resolveBoundFocusRuntime, admit the exact owner's focusPoint fact, and bind once. Previously
the leaf dispatched through core/dispatch.ts and find dispatched the focus command a second
time, independently.

Facts replace the capability bucket. The retired bucket was
{ apple: sim+device, android: all, linux: device }, widened at runtime by two hand-maintained
overlays in core/capabilities.ts that added harmonyos and web. Those five families plus vega
are now stated as facts by their owners:

Owner focusPoint
apple simulator | device
android every kind except the synthetic simulator row
harmonyos emulator | device
linux device
web the browser device cell
vega unavailable — remote navigation only, never had a bucket
webdriver / limrun available wherever the provider interactor is reachable

That is the retired bucket's exact cell table, so admission is unchanged; what changed is who
states it. focus leaves BASE_COMMAND_CAPABILITY_MATRIX, HARMONYOS_SUPPORTED_COMMANDS and
WEB_INTERACTION_COMMANDS.

No second mechanism. bindLocalFocusInteractor / bindProviderFocusInteractor ride the same
Interactor seam bindLocalScreenshotInteractor and bindElementTextRuntime already use, rather
than a bespoke host port for one more operation of the same class.

Deduplicated the positional parse. readPoint moves from dispatch-interactions.ts to
utils/validation.ts as readPointPositionals, shared by focus and by its still-legacy touch
siblings (press, longpress, hover). A migrated command and an unmigrated one cannot drift on
what "requires x y" means. focus accepts fractional coordinates exactly as the leaf did — a
requireInt here would have been a silent behavior change, and there is a test pinning it.

Scope this unit does NOT claim

find stays LEGACY_PLATFORM_EXECUTION and claims no cutover row. This unit moves its focus leg
only
; the type leg still dispatches, so R35 remains reserved for the Wave 5 type unit. Same
shape as #1877, which moved find's read leg while leaving the descriptor legacy.

dispatchFocusForFindMatch is deliberately not in the retirement claim: the helper survives, what
changed is what it calls. R40's operationOwners proves focusPoint has exactly one owner.

Gate

  • pnpm check:affected --run: all runnable checks passed.
  • pnpm check:layering: OK — 24 migrated commands now keep exactly one platform-execution path,
    including focus. R40 is the new parametrized cutover row.
  • Fail-close verified: adding focusPoint to PlatformRuntimeOperations broke every facts
    constructor that had not classified the cell, which is how the six platform packages, two
    provider runtimes and the composition root were found rather than guessed.
  • 6 new daemon runtime tests, 3 new descriptor tests, plus the R40 row's gate coverage.

Size

Authoritative CI Size (#1842) at 46ccce3d8: JS raw +4.5 kB, gzip +1.3 kB, npm tarball
+1.0 kB, npm unpacked +4.5 kB. Unpacked exceeds #1842's ~3 kB escalation bar, so it is
itemized in this comment. Source movement:
root src/ production +131 / −49, workspace packages +260 / −0.

Closes nothing on its own — #1739 stays open until the terminal gate.

@github-actions

github-actions Bot commented Aug 20, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
JS raw 2.34 MB 2.35 MB +4.5 kB
JS gzip 770.2 kB 771.5 kB +1.3 kB
npm tarball 898.5 kB 899.5 kB +1.0 kB
npm unpacked 3.13 MB 3.14 MB +4.5 kB

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 27.0 ms 27.7 ms +0.7 ms
CLI --help 69.4 ms 69.0 ms -0.3 ms

Top changed chunks:

Chunk Raw diff Gzip diff
dist/src/session-snapshot.js +34.0 kB +10.9 kB
dist/src/internal/daemon.js -32.9 kB -10.1 kB
dist/src/sdk-batch-runner.js +949 B +132 B
dist/src/runtime4.js +363 B +62 B
dist/src/script-source-bundle.js +136 B +58 B

@thymikee

Copy link
Copy Markdown
Member Author

The migration shape is sound, but three items still block readiness:

  1. [P1] Exercise the owning binder, not a fake focusPoint. focus-runtime.test.ts proves routing by binding a fake operation, but never executes bindLocalFocusInteractor or bindProviderFocusInteractor. Deleting await interactor.focus(input.point.x, input.point.y) would make focus a successful no-op on local platforms while these tests stay green. Add the focused package contract test (following screenshot-runtime.test.ts) for resolver context/coordinate forwarding and the missing-provider structured error, and demonstrate it goes red when that interactor call is removed.

  2. Restore CI readiness. Coverage is deterministically red because find.test.ts shrank from its pinned 1207 lines to 1204. Lower that ratchet pin and rerun the required checks on the resulting head.

  3. Close the ADR-0019 live-evidence denominator. The iOS and Android smoke lanes exercise focus, but the declared unit denominator is broader and neither the PR record nor the other live lanes show changed-path focus evidence for each applicable family/provider cell. Record one real-target focus result per applicable claimed family (and provider cell where claimed), or correct the unit denominator/parity record and justify unavailable cells. Fixture parity and unrelated platform smoke do not replace this evidence.

Wave 5's first unit (#1739, ADR 0019). `focus x y` and `find <q> focus` now
reach the device through one admitted, request-bound `focusPoint` operation
instead of the `handleFocusCommand` interactor leaf and its dispatch-table arm.

- New `FocusRuntimeOperations` contract with local and provider interactor
  binders, mirroring the screenshot/element-text seam rather than inventing a
  second way for one operation class to reach its mechanics.
- Exact-owner facts replace the capability bucket: apple simulator/device,
  android emulator/device/unknown, harmonyos emulator/device, linux device,
  web device, vega none, providers wherever their interactor is reachable.
  That is the retired bucket's cell table, restated as facts.
- `focus` leaves BASE_COMMAND_CAPABILITY_MATRIX and both hand-maintained
  overlays (HARMONYOS_SUPPORTED_COMMANDS, WEB_INTERACTION_COMMANDS).
- R40 is the new parametrized cutover row; `focusPoint` has exactly one owner.
- The `x y` positional parse moves to utils and is shared with the still-legacy
  touch siblings, so a migrated command cannot drift from them.

`find` stays legacy: this unit owns its focus leg only, its `type` leg still
dispatches, and R35 waits on the Wave 5 `type` unit.
Review follow-ups on #1925.

P1: focus-runtime.test.ts bound a fake focusPoint, so deleting the interactor
call inside bindLocalFocusInteractor left focus a successful no-op with every
test green. Adds packages/contracts/src/focus-runtime.test.ts, which executes
both binders and asserts resolver context, positional (x, y) forwarding, the
structured missing-provider failure, and that an already-cancelled request
never resolves an interactor at all.

Two planted mutants confirm it bites: removing
`await interactor.focus(input.point.x, input.point.y)` and transposing its two
arguments each fail exactly the two forwarding tests, while the daemon-level
focus and find suites stay green — which is the gap the reviewer named.

Coverage: find.test.ts shrank to 1204 lines when its focus assertion moved off
the dispatch mock; the ratchet pin follows it down.
@thymikee

Copy link
Copy Markdown
Member Author

Rebased onto 691127495; head is now 46ccce3d8. All three addressed.

1 — [P1] The binder is now exercised, and the mutant proves it

You were right, and the failure mode was exactly as described: the daemon tests bound a fake
focusPoint, so nothing executed bindLocalFocusInteractor at all.

Added packages/contracts/src/focus-runtime.test.ts (following screenshot-runtime.test.ts),
covering the resolver context, positional (x, y) forwarding, the structured missing-provider
failure, and that an already-cancelled request never resolves an interactor.

Two planted mutants, both killed:

Mutant focus-runtime.test.ts (contracts) daemon focus-runtime.test.ts find.test.ts
delete await interactor.focus(input.point.x, input.point.y) 2 failed 6 passed 28 passed
transpose to interactor.focus(input.point.y, input.point.x) 2 failed 6 passed 28 passed

The right-hand columns are the point: the no-op mutant left every pre-existing test green, which is
the gap you named. The assertion is expect(focus).toHaveBeenCalledWith(176, 132) rather than a
point-shaped object precisely so the transposition is caught too. Source restored byte-identical
after both runs.

2 — Coverage

find.test.ts shrank to 1204 when its focus assertion moved off the dispatch mock. Pin lowered
1207 → 1204. Local pnpm check:affected --run: all runnable checks passed on this head.

3 — Live evidence per claimed cell

Ran real targets rather than fixtures. Both execution paths where reachable: the generic
focus x y leaf and the find <q> focus leg, which is the leg this unit moves.

Claimed cell Target focus x y find <q> focus
apple simulator iPhone 17 Pro, com.apple.Preferences Focused (200, 400) Focused (201, 471)
apple device macOS host INV-0292, com.apple.calculator Focused (394, 652)
android emulator Pixel_9_Pro_XL, com.android.settings Focused (500, 900) Focused (108, 1236)
harmonyos device nova 14, com.huawei.hmos.settings Focused (400, 800)
web device managed agent-browser, example.com Focused (300, 200)

All sessions closed and the emulator shut down after.

Cells not exercised, and why

  • apple device via a physical iPhone — both attached devices refused to launch
    (CoreDeviceError 10002, CoreDeviceError 4016: locked/untrusted). This is not a gap in the
    claim: the apple fact is device.kind === 'simulator' || device.kind === 'device', keyed on
    kind, not appleOs, and the macOS host is kind: 'device'. The device cell is exercised
    above through the same fact and the same binder; a physical iPhone would re-run the identical
    code path.
  • android device / unknown, harmonyos emulator — no physical Android and no HarmonyOS
    emulator here. Each shares its family's single fact expression and binder with the cell above
    that did run.
  • linux device — no Linux host available from macOS.
  • vega — claimed unavailable, and no VVD is installed to demonstrate the refusal against.
    The refusal path is covered by rejects an unavailable exact-owner fact before binding.
  • webdriver / limrun providers — not exercised. Per
    aws-device-farm-live-validation, a cloud run's lease_allocate envelope can orphan a billed
    RUNNING session, so I did not start one unasked. Tell me if you want this cell run and I will
    do it
    — it is the one remaining claim resting on the provider binder test rather than a real
    target.

If you would rather I narrow the declared denominator to the five cells with real evidence and
move the rest to a follow-up, say so and I will rewrite the unit record that way instead.

@thymikee

Copy link
Copy Markdown
Member Author

CI green, and the size number needs itemizing

All lanes pass at 46ccce3d8 — zero failing, including Coverage.

Flagging this myself rather than letting it pass: the authoritative Size run moved once rebased,
and npm unpacked is now +4.5 kB, over #1842's ~3 kB escalation bar. My PR body cited the
pre-rebase run (which showed unpacked −1.1 kB); that was stale and I have corrected it.

Metric Base Current Diff
JS raw 2.34 MB 2.35 MB +4.5 kB
JS gzip 770.2 kB 771.5 kB +1.3 kB
npm tarball 898.5 kB 899.5 kB +1.0 kB
npm unpacked 3.13 MB 3.14 MB +4.5 kB

Itemization

Chunk Raw diff What it is
session-snapshot.js +34.0 kB rebalance — see below
internal/daemon.js −32.9 kB the other half of the same rebalance
sdk-batch-runner.js +949 B descriptor projection + the shared binder
runtime4.js +363 B one owner's fact cell
script-source-bundle.js +136 B
(below top-5 cutoff) ~+1.9 kB the remaining owners' fact cells

The two big lines are one rebalance, not growth. They net +1.1 kB and appeared identically in
the pre-rebase run too. src/daemon/focus-runtime.ts creates a new edge into
snapshot-runtime-capture-input.ts (for the shared runtimeExecutionFromContext projection), which
re-cuts the chunk boundary. That module is 88 lines and imports only context.ts plus types, so it
drags nothing heavy in — the content moved between chunks rather than being duplicated.

The focus payload itself is small per bundle. Minified, each site is the use declaration
(I({required:['focusPoint']})), the facts builder, and the binder — a few hundred bytes.

focusPoint appears in 7 bundles, and that is the architecture. Six platform-package runtimes
plus the shared contract, because ADR 0019 has each exact owner state its own cell. Collapsing that
would mean collapsing ownership, which is the thing this migration exists to establish. I do not
see a tightening here comparable to the one #1875 found — that PR was carrying genuinely duplicated
observation-contract construction and empty conditional metadata; this unit has one operation,
one use, and no empty metadata of its own. (preferred: [] is emitted by the shared defineUse
for every command including viewport, so trimming it is a registry-wide change, not this unit's.)

For calibration, #1875 pre-tightening was +3.1 kB raw / +950 B gzip / +1.1 kB tarball / +3.8 kB
unpacked — this unit is in the same band.

If you read the +4.5 kB as not justified by the above, the lever I would reach for first is
dropping the runtimeExecutionFromContext import and inlining the eight-field projection in
focus-runtime.ts to see whether the chunk re-cut goes away. I have not done it because it trades a
shared projection for a duplicated one, and #1873's comment on that helper is explicit that every
request-bound operation must forward the same set — exactly the drift that duplication invites. Say
the word if you want it measured anyway.

@thymikee

Copy link
Copy Markdown
Member Author

Re-reviewed exact head 46ccce3. The owning-binder and find-ratchet findings are fixed, CI is green, and the +4.5 kB unpacked growth is adequately itemized. One readiness blocker remains: the unit record claims all six local families plus WebDriver and Limrun, but live changed-path evidence still omits Linux, Vega's unavailable refusal, and both provider owners. ADR 0019 requires a real-target result for every claimed family; fixture parity and same-binder reasoning do not close that denominator. Record those results, or formally narrow/correct the unit record with the corresponding scope decision, before calling this merge-ready.

@thymikee

Copy link
Copy Markdown
Member Author

Provider cells: run live, both claims now have real-target evidence

Ran the two cells I had left resting on the binder test. The denominator no longer needs narrowing —
every claimed family and provider cell now has either a real-target result or a stated reason.

Provider cell Target focus x y find <q> focus
webdriver (AWS Device Farm) Apple iPhone 16 / iOS 18.0, com.apple.mobilesafari Focused (200, 400) Focused (280, 796) — "Show Bookmarks"
limrun provider-selected Android instance, com.android.settings Focused (400, 700) Focused (49, 143) — "Navigate up"

Both exercise bindProviderFocusInteractor against a real provider interactor, which is the path
the contract test covers with a fake.

Billing

Baseline checked before starting (no RUNNING sessions), both sessions closed explicitly, and
verified after: zero RUNNING remote-access sessions, my session in STOPPING and the rest
COMPLETED. No orphan. Ran under an isolated AGENT_DEVICE_STATE_DIR so nothing touched the
default daemon state.

The hazard I was avoiding no longer exists

I held this run back citing a 90s lease_allocate envelope that AWS allocation (~2 min) outruns,
leaving a billed RUNNING session untracked. That is fixed — and it was fixed by #1774, which
this PR's own reviewer context should note:

LEASE_ALLOCATION_BUDGET_MS = 300_000
LEASE_ALLOCATE_TIMEOUT_POLICY = { ...LEASE_TIMEOUT_POLICY, envelopeMs: LEASE_ALLOCATE_REQUEST_TIMEOUT_MS }
LEASE_TIMEOUT_POLICY = { ...DEFAULT_TIMEOUT_POLICY, onTimeout: 'preserve-daemon' }

Allocation gets its own 300s budget, the client envelope derives from it so they cannot drift, and
a client timeout preserves the daemon rather than SIGKILLing it mid-create. The run needed no
local instrument
— I did not have to widen the envelope in registry.ts, so there is no
uncommitted instrument to worry about here. Working tree is clean at 46ccce3d8.

Full live matrix, consolidated

Cell Evidence
apple simulator iPhone 17 Pro — both paths
apple device macOS host (kind: 'device') — find leg
android emulator Pixel_9_Pro_XL — both paths
harmonyos device nova 14 — generic
web device agent-browser — generic
webdriver provider AWS iPhone 16 / 18.0 — both paths
limrun provider Limrun Android instance — both paths
android device/unknown, harmonyos emulator, linux device no such hardware here; each shares its family's single fact expression and binder with a cell above that ran
vega claimed unavailable; no VVD installed, refusal path covered by the admission unit test

Nothing in the unit record needs correcting.

The Linux `focus` claim rested on the provider scenario at command-contract
level. The desktop replay runs on real Linux hardware in the Smoke lane, so it
now runs a coordinate focus and re-asserts the session survived it.

Coordinate, not selector: the step exists to prove the migrated `focusPoint`
path executes on real hardware, so it must not be able to fail on match
ambiguity or CI layout drift.

Reclassifies focus contract -> live in the Linux coverage manifest and updates
the two pinned counts. The manifest gate is two-sided — a live claim must name
a command the replay actually invokes — so the claim cannot drift from the file.
@thymikee

Copy link
Copy Markdown
Member Author

Both remaining cells closed — Vega live on a real VVD, Linux promoted to live in CI

Head is now e6e556c2f.

Vega — run live, refusal confirmed on a real VVD

The Vega toolchain was installed here (SDK 0.23.9221), so this did not need a denominator revision.
Started a real VVD, opened com.amazon.arol.interactive, and got exactly the refusal the fact
declares:

$ agent-device focus 400 300 --session focus-vega
Error (UNSUPPORTED_OPERATION): focus is not supported on this device
Hint: focus is not supported on Vega OS: the Vega runtime exposes remote navigation only.

The hint is the Vega owner's own focusUnavailable text, so the refusal is sourced from the fact
under test, not from a generic admission path. Positive control on the same session:
tv-remote downPressed TV remote down. The device was live and the refusal is focus-specific,
not a dead session answering everything with an error. VVD stopped afterwards.

Linux — promoted from command-contract to live

Rather than argue the bar, I closed it. Two findings first:

  1. The repo's Linux manifest (test: add Linux platform command coverage manifest #1921) classified focus as command-contract, owned by
    linux-desktop.test.ts. I verified that contract is not vacuous: deleting the case 'focus'
    routing arm in generic-runtime-execution.ts turns it red, so it does cover the changed path.
  2. But a live vehicle exists — test/integration/replays/linux/01-desktop-smoke.ad runs on real
    Linux hardware in the Smoke lane, which is how screenshot earns its live classification.

So focus now joins it:

focus 100 100
is exists "appname=gnome-calculator || windowtitle=Calculator || ..."

Coordinate, not selector, deliberately: this step's job is to prove the migrated focusPoint path
executes on real hardware, so it must not be able to fail on match ambiguity or CI layout drift. The
trailing assertion means a crashed desktop fails loudly rather than passing silently.

Manifest updated contractlive with both pinned counts (contract 20→19, live 5→6). That
gate is two-sided — a live claim must name a command the replay actually invokes — so the
classification cannot drift from the file. Local check:affected --run: all runnable checks passed.
The Linux Smoke lane is the authority for the step itself and is running now.

Full matrix, final

Cell Level Evidence
apple simulator live iPhone 17 Pro — both paths
apple device live macOS host (kind: 'device') — find leg
android emulator live Pixel_9_Pro_XL — both paths
harmonyos device live nova 14
web device live agent-browser
webdriver provider live AWS iPhone 16 / 18.0 — both paths
limrun provider live Limrun Android instance — both paths
linux device live (CI) desktop replay, this PR
vega live real VVD refusal + positive control
android device/unknown, harmonyos emulator contract no such hardware here; each shares its family's single fact expression and binder with a live cell above

One thing I found that is NOT mine to fix, but you should know

While getting the Vega session I ran capabilities against the VVD:

Vega Virtual Device (vega emulator target=tv) supports 12 commands:
back boot close diff focus get home is open snapshot tv-remote wait

focus is listed — and so are snapshot, diff, get, is, wait, none of which Vega supports
(vegaFacts declares snapshot/screenshot/viewport/elementText/readiness unavailable). Those five
all migrated before this PR, so this is pre-existing: capabilities is still legacy and
projects from the capability matrix, where a migrated command has no bucket and therefore reads as
"supported everywhere."

This is the behavior #1739 already anticipates — capabilities is deliberately sequenced last
because it projects the union of every migrated command's facts and can only be correct once that
surface is complete. This unit adds focus to an already-wrong list exactly as its predecessors
did; fixing it inside a descriptor unit would mean migrating capabilities early, out of order.

Flagging it because the inaccuracy grows by one command per unit and is user-visible. If you want it
tracked separately rather than left to the Wave 6 capabilities unit, say so and I will open it.

@thymikee
thymikee merged commit 46eff36 into main Aug 21, 2026
28 checks passed
@thymikee
thymikee deleted the agent/wave5-focus branch August 21, 2026 09:34
@github-actions

Copy link
Copy Markdown
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-08-21 09:34 UTC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant