Skip to content

Simplify Bell's browser integration with Stagehand tools - #33

Open
shrey150 wants to merge 2 commits into
mainfrom
codex/bell-stagehand-middleware
Open

shrey150 wants to merge 2 commits into
mainfrom
codex/bell-stagehand-middleware

Conversation

@shrey150

@shrey150 shrey150 commented Oct 27, 2026 •

Copy link
Copy Markdown

Use Stagehand's shared browser tools for Bell's action batches, page snapshots, and screenshots. Bell keeps its typed tool inputs and browser lifecycle.

  • Reuse one set of browser tools per attached Stagehand client, retaining Bell's session release, CAPTCHA gates, and credential protections.
  • Keep mixed CSS/snapshot-ID actions in one batch, encode input values as literals, and read the live URL after keyboard-triggered navigation.
  • Keep the browser tools dependency external in Eve and Next.js server builds.

E2E Test Matrix

Verified head 8cbe0a0 against base 89954b6. Full-agent runs used GPT-5.6 Sol through Vercel AI Gateway and real Browserbase sessions. Targets were public or synthetic.

Command / flow Observed output Confidence / sufficiency
pnpm check in a clean checkout 347 tests passed; 61 opt-in tests skipped. Typecheck, lint, Eve build, and Next production build passed. Repository regressions and compilation; live checks below are separate.
PostgreSQL-backed OAuth/store/privacy suites 43/43 passed against disposable PostgreSQL 16. Real database locking and persistence; external OAuth responses were synthetic.
Bell web UI: starter request, follow-up, screenshot, New conversation Real model response and rendered PNG on both turns; one hosted browser reused; reset cleared UI and provider confirmed release. Actual browser UI → Eve → model → browser → UI flow.
Paired Bell HTTP agent tasks: page facts/PNG, form submission, two-turn form persistence; two repeats each Base 6/6 and PR 6/6 task outcomes passed across 16 total model turns. Independent DOM assertions verified literal text, dropdown, checkbox, confirmation, and persisted state. Small comparison using the same model/provider, with real tool execution. All 12 browsers released and temporary Contexts deleted.
Production bundle: prewarm build, built Eve server plus Next server Full agent turn passed in 10.261 s; navigation, snapshot, and 92,047-byte PNG succeeded with no tool errors; reset and provider release passed. Exercises packaged external middleware in a full production-mode agent turn. Local startup prerequisites are described below.
BELL_LIVE_MIDDLEWARE=1 middleware test, local and hosted 1/1 passed per provider. All seven actions, mixed selectors/IDs, literal values, stale IDs, URL metadata, PNG, reuse, cleanup.
Hosted Context persistence, CAPTCHA detection, proxy checks 3/3 passed. Saved cookie restored after browser release; Context deleted. Proxy check verifies requested configuration and live navigation, not independently measured geography.
Real Chrome synthetic checkout and Autofill extension suites 7/7 checkout checks and 3/3 extension checks passed. Privacy, cancellation, extension fill/tab selection; synthetic credentials and payment fields, no purchase.
Hosted synthetic password/OTP login followed by middleware actions Passed, including rejected-host protection and credential redaction. Secure login and middleware continue in one browser; no actual 1Password vault involved.
Cross-process restart, base and PR Both fail: reinitialization reports an already initialized Stagehand instance; replacement browser loses the original page. Existing failure reproduced on both revisions.
Cancel during a three-by-10-second wait batch, base and PR Cancel acknowledged in 19/18 ms; Ready arrived after 28.736/28.797 s. Following snapshot did not run; both browsers released. Cancellation is cooperative and slow on both revisions.
Real reCAPTCHA Initial default-timeout runs failed on base and PR. PR retries passed with default and extended timeouts, including visible verification success. Intermittent provider-dependent result; not an unconditional pass.

Paired results and limits

Metric Base PR
Completed task outcomes 6/6 6/6
Tool calls 34 36
Recovered tool errors 0 1
Mean task duration 18.52 s 17.45 s
Input / output tokens 317,213 / 1,638 309,024 / 1,555

The PR recovered one model-selected ambiguous CSS selector by narrowing it, then completed the form correctly. Timing excludes verification/cleanup. These few cached, stochastic runs support compatibility for the tested flows; they do not establish a general latency, cost, or quality improvement.

Local production setup and remaining coverage

next start alone leaves the Eve proxy without a backend. Run the built Eve server alongside Next for a local production check. The normal build script uses --skip-sandbox-prewarm; a full production agent turn failed until pnpm exec eve build provisioned the sandbox template, followed by node scripts/package-browser-assets.mjs. These start/build commands are unchanged by this PR.

Both development instances used Eve's automatic workspace setup. Stagehand, Browserbase SDK, Playwright, Eve, and model-provider versions matched; the PR added middleware 0.1.1.

Real 1Password vault login and real Link payment execution remain unverified. Test sessions, temporary Contexts, local servers, and the disposable database were cleaned up.

@shrey150
shrey150 requested a review from a team as a code owner October 27, 2026 22:48
@socket-security

socket-security Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedstagehand-middleware@​0.1.1781008588100

View full report

@shrey150 shrey150 changed the title Use stagehand-middleware for Bell browser tools Simplify Bell's browser integration with Stagehand tools Oct 27, 2026
@shriyatheunicorn
shriyatheunicorn self-requested a review October 29, 2026 06:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant