Benchmark report

Task verified

7m 50s elapsed

The saved attempt met the documented success criterion.

Experience score

83/ 100

Needs Alignment
  • AInstall
  • AAuth
  • SExecute
  • BDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth A, Execute S, Docs B, Current SAInstallAAuthSExecuteBDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Documentation hosts were unreachable, so the agent reverse-engineered the API from installed SDK source and probed three times to find the CDP WebSocket auth header.

Where the time went

7:50 agent time
  • Research0:23
  • Attempt4:40
  • Verify2:47
7m 50sTotal elapsed
—Time to first success
0:00Waited on the owner

Judge reason

Transcript shows a single real run of the saved script /tmp/benchmark-home/hyperbrowser/cookie_profile_test.py (full source captured at transcript line 111). It uses documented SDK ops: profiles.create (POST /api/profile) -> printed profile e4828d97-3458-4200-a9eb-af5eedc26e67; sessions.create with profile={id, persistChanges:True} -> session A c7ed38a2-8e92-44b9-97c9-9162c1d16f29 and, after A was stopped, session B 75b21259-0e1b-492f-8fba-87d38b7a82b3 (distinct, same profile). Playwright connect_over_cdp used the wsEndpoint with the query string stripped and the credential sent as the x-hyperbrowser-token request header (header name learned from the live 401 'missing Authorization header, x-hyperbrowser-token header, or token query param'); API key sent only via the SDK's x-api-key header. Run stdout (line 116) shows session A loading httpbin cookies/set (final URL https://httpbin.org/cookies, 200) returning {"cookies":{"tpc":"benchmark"}} and context cookie tpc=benchmark; sessions.stop -> {'success': True} polled to status closed (server endTime 1793747819851 = 23:16:59.851Z) before B was created; session B's httpbin.org/cookies returned {"cookies":{"tpc":"benchmark"}} -> cookieSurvived true. finally block: A already closed, B stop {'success': True} (endTime 23:17:04.627Z), profiles.delete {'success': True}. Printed values match the saved out/result.json and out/06_session_b_visit.json shown at line 126 and the copy at /workspace/home/result.json; a grep -rF of the key over the home dir reported 'no key in home' and saved URLs show token=%3CREDACTED%3E. Authenticity is corroborated by server-side endTime epochs falling inside the local run window, earlier failed probes (401 WebSocket, empty profile list, differing dp-us-west-1/us-east-1 hosts) and real httpbin output - nothing mocked. Note: the preserved snapshot /workspace/home contains only result.json and work/ (docs/auth.md, venv); the hyperbrowser/ code and out/*.json existed at run end per the listing at line 131 but were not captured, so those artifacts were verified from the transcript rather than the snapshot.

Attempt record

  1. 2:15Answered from saved key: Hyperbrowser API keyno wait
  2. —First success verified

Attempt artifacts saved.