Skip to content

feat: reliable browser challenge handoff and optional local Chrome - #645

Merged
davidmckayv merged 6 commits into
mainfrom
david/openbot-browser-handoff
Sep 26, 2026
Merged

davidmckayv merged 6 commits into
mainfrom
david/openbot-browser-handoff

Conversation

@davidmckayv

Copy link
Copy Markdown
Contributor

Browser challenges could strand a Bot or let it continue with stale page state. This change detects Cloudflare Challenge Pages and conservative visible challenge signals, pauses computer actions, and lets a person take over the existing page. Handback resumes the exact request after a fresh snapshot; cancellation, expiry, and browser interruption remain distinct outcomes.

The viewer reconnects without releasing human control, and a reloaded conversation offers an explicit Resume task action for its pending browser tool. Managed browsing now uses full Chromium. Local API deployments can opt into installed Chrome with separate per-Bot profiles, a loopback endpoint, the browser sandbox retained, and shell execution disabled. Setup is documented in docs/configuration.md.

Validation:

  • Repository CI suite: 5,290 passed, 36 existing opt-in skips, zero failures; affected follow-up regressions passed.
  • App, server, and computer typechecks; scoped formatting/lint; app production build and server build passed.
  • Actual full headless Chromium navigated, read, and snapshotted example.com. Actual installed Chrome rendered in the live viewer.
  • The selected General Assistant rendered a real chart and used its browser tool. A synthetic challenge completed through detection, takeover, human input, handback, and a fresh browser read of the success phrase.
  • Reloaded the app during a waiting challenge: the exact request and browser page survived, Resume task continued the original tool call, and the Bot read the completed page successfully. Fresh-snapshot enforcement remained set after handback until Resume task captured the new state.

Detection is deliberately conservative. Some embedded or score-only challenges are not automatically identifiable; model-requested handoff remains available. These checks establish the browser and handoff behavior, not guaranteed acceptance by Cloudflare, Google, or any other website.

@davidmckayv
davidmckayv merged commit 01c3ea3 into main Sep 26, 2026
17 checks passed
@davidmckayv
davidmckayv deleted the david/openbot-browser-handoff branch September 26, 2026 23:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant