The Prompting Company

Benchmark report

Task verified

5m 10s elapsed

The saved attempt met the documented success criterion.

Experience score

88/ 100

Agent-Ready
  • SInstall
  • AAuth
  • SExecute
  • ADocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth A, Execute S, Docs A, Current ASInstallAAuthSExecuteADocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
Missing/404 Connect API docs and a quickstart silent on WebSocket auth forced host probing and a failed header-auth CDP attempt before falling back to connectUrl.

Where the time went

5:11 agent time
  • Research0:16
  • Attempt3:21
  • Verify1:33
5m 10sTotal elapsed
2:36Time to first success
0:00Waited on the owner

Judge reason

Saved run_session.py (no mocks) does POST https://api.browserbase.com/v1/sessions with X-BB-API-Key header; artifacts/create_session_response.json shows HTTP 201 with id 56e64d66-af7d-4582-ae24-2b25223fb74f, status RUNNING, region us-west-2, live Ratelimit/Request-Id headers and connectUrl wss://connect.usw2.browserbase.com/?signingKey=REDACTED. Playwright connect_over_cdp used that documented connectUrl (header-only auth was tried first and rejected by Browserbase with 401 'either apiKey or signingKey must be provided', logged in result.json); run_output.txt prints '[page] title : Example Domain' with page HTTP 200. Termination runs in a finally block via the documented Update a Session POST /v1/sessions/{id} {"status":"REQUEST_RELEASE"} -> HTTP 200 with status COMPLETED and endedAt 2026-11-03T21:53:45.481+00:00 (terminate_session_response.json). Printed values match the saved responses; the API key was passed only via BB_KEY_FILE path and the X-BB-API-Key header, never in a URL, and a grep of the real key across the home dir found no leak (all artifacts show REDACTED). Script exited 0.

Attempt record

  1. 1:40Answered from saved key: Browserbase API keyno wait
  2. —First success verified

Attempt artifacts saved.