Skip to content

feat(extension): add authenticated remote gateways and background task tabs - #227

Open
lyingbug wants to merge 1 commit into
Tencent:mainfrom
lyingbug:feat/remote-extension-connection
Open

feat(extension): add authenticated remote gateways and background task tabs#227
lyingbug wants to merge 1 commit into
Tencent:mainfrom
lyingbug:feat/remote-extension-connection

Conversation

@lyingbug

Copy link
Copy Markdown

Problem and behavior

A server-hosted agent needs to operate a user's already logged-in Chrome while the user continues chatting in another tab. The extension currently connects to a local daemon, and using an automation request for the chat preview can compete with a long-running command or human-help wait.

This PR adds an opt-in authenticated gateway connection and background task tabs for that mode. It addresses the extension-side portion of #50. The native CLI/daemon and published automation RPC payloads are unchanged; this does not add remote CLI transport or expose the daemon publicly.

Companion gateway and product integration: Tencent/WeKnora#3180

Changes

  • Add a Remote connection section to the popup, showing the destination before connecting. A one-use pairing link is exchanged for a renewable device grant. The credential stays in trusted extension-local storage and an authenticated WebSocket subprotocol, outside the URL/query string. Remote WSS is required; plaintext WS is limited to loopback. A saved remote connection never silently falls back to localhost.
  • Persist rotation candidates before renewal so response loss can be retried. Reuse the existing connection controller, cleanup and reconnect behavior when changing endpoints. Local-daemon mode remains the default.
  • In remote mode, create inactive tabs inside a labeled group in an existing normal window. Logical target selection, human-help requests and borrow requests do not proactively activate a tab/window. Existing tabs retain the original explicit borrow approval, but are borrowed in place and released without moving/closing them.
  • Enforce ownership by concrete created/borrowed tab IDs, including passive reads. A shared window or visual group never grants access to unrelated tabs. Stopping a task closes only its created tabs. Local-daemon sessions retain dedicated Agent Windows.
  • Add optional gateway.task_focus and gateway.task_preview messages scoped to an existing session. Preview captures via CDP outside the automation queue, coalesces concurrent captures, and bounds the encoded JPEG width to 640 physical pixels including HiDPI displays. Focus is intended for an explicit user action.

The manifest adds tabGroups permission for task identification. Remote mode does not resize the shared user window. The visible group label defaults to BrowserSkill; gateways may supply a bounded service_name during authorization.

Proposed contracts

docs/remote-extension-connection.md specifies the HTTP authorization exchange, credential-bearing subprotocol and optional UI messages. These are proposals for upstream review, not previously published BrowserSkill standards. The gateway authenticates users, maps each request to their existing task, rate-limits preview and closes revoked connections. No cloud gateway implementation is included in this repository.

Validation

  • Full extension suite: 1,269 passed, 40 skipped across 94 passing test files.
  • biome check .: 370 files checked, no issues.
  • Extension tsc --noEmit and wxt zip: passed.
  • New regression coverage: endpoint validation and credential binding, storage change ordering, retryable credential rotation, task-tab isolation/cleanup, borrowing without moving tabs, human-help without focus, preview coalescing and physical HiDPI bounds.
  • Companion integration test with the unmodified released v0.2.1 daemon and isolated headed Chromium: popup pairing, navigation, Chinese text input, click, snapshot, background tab group, active tab/window preservation, preview during human-help waits, explicit focus, pause/resume and stop. This test runs from the linked WeKnora PR; it is not an extension-only unit test.
  • The consumer build also applied this exact patch to a clean pinned upstream checkout and built the extension with frozen dependencies.

Rust/Windows and the unrelated DSH plugin suites were not rerun locally because this PR changes the extension and its translations only. Existing skipped extension tests remain skipped. Production gateway deployment and capacity are outside these local checks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant