Benchmark report

Task verified

6m 22s elapsed

The saved attempt met the documented success criterion.

Experience score

84/ 100

Needs Alignment
  • SInstall
  • AAuth
  • SExecute
  • BDocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth A, Execute S, Docs B, Current ASInstallAAuthSExecuteBDocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
Determining whether the API key could ride as a header on the CDP WebSocket: docs were silent/404, forcing ~40 transcript steps of probing before using the documented connectUrl.

Where the time went

6:22 agent time
  • Research0:25
  • Attempt4:21
  • Verify1:36
6m 22sTotal elapsed
3:24Time to first success
0:00Waited on the owner

Judge reason

Saved run.mjs creates a session via the documented POST https://api.browserbase.com/v1/sessions with the key only in the X-BB-API-Key header; session-create-response.json shows http_status 201, id 899b4084-5688-4e71-904b-880258815435 and connectUrl wss://connect.usw2.browserbase.com/?signingKey=<REDACTED>. Playwright (playwright-core 1.64.1, installed per work/package-lock.json) connects via chromium.connectOverCDP(connectUrl) with the X-BB-API-Key header, navigates to https://httpbin.org/forms/post and fills/submits the form; echoed-response.json and run-output.txt show the live echo from https://httpbin.org/post with form.custname 'Benchmark Test', form.custtel '555-0100', remote egress origin 52.27.44.164 and Chrome/156 UA (httpbin was unreachable locally, exit 56, so the fetch demonstrably came from the remote browser). Termination runs in the finally block via POST /v1/sessions/{id} {"status":"REQUEST_RELEASE"}: http_status 200, status COMPLETED, endedAt 2026-11-03T22:25:57.074+00:00. Transcript entry for the single run matches the saved files byte-for-byte, no mocking or hard-coded data. A grep of the key across the home dir found no match and my own scan for key/JWE patterns in saved files found none; the connectUrl query carries only the short-lived per-session signingKey, redacted everywhere, not the API key. NOTES-cdp-auth.md records a measured 9-header probe showing the CDP gateway rejects header-only auth (401 'either apiKey or signingKey must be provided'), justifying use of the documented connectUrl.

Attempt record

  1. 1:59Answered from saved key: Browserbase API keyno wait
  2. —First success verified

Attempt artifacts saved.