Benchmark report
Task verified
7m 50s elapsed
The saved attempt met the documented success criterion.
83/ 100
Needs Alignment- AInstall
- AAuth
- SExecute
- BDocs
- SCurrent
Documentation hosts were unreachable, so the agent reverse-engineered the API from installed SDK source and probed three times to find the CDP WebSocket auth header.
Where the time went
7:50 agent time- Research0:23
- Attempt4:40
- Verify2:47
Judge reason
Transcript shows a single real run of the saved script /tmp/benchmark-home/hyperbrowser/cookie_profile_test.py (full source captured at transcript line 111). It uses documented SDK ops: profiles.create (POST /api/profile) -> printed profile e4828d97-3458-4200-a9eb-af5eedc26e67; sessions.create with profile={id, persistChanges:True} -> session A c7ed38a2-8e92-44b9-97c9-9162c1d16f29 and, after A was stopped, session B 75b21259-0e1b-492f-8fba-87d38b7a82b3 (distinct, same profile). Playwright connect_over_cdp used the wsEndpoint with the query string stripped and the credential sent as the x-hyperbrowser-token request header (header name learned from the live 401 'missing Authorization header, x-hyperbrowser-token header, or token query param'); API key sent only via the SDK's x-api-key header. Run stdout (line 116) shows session A loading httpbin cookies/set (final URL https://httpbin.org/cookies, 200) returning {"cookies":{"tpc":"benchmark"}} and context cookie tpc=benchmark; sessions.stop -> {'success': True} polled to status closed (server endTime 1793747819851 = 23:16:59.851Z) before B was created; session B's httpbin.org/cookies returned {"cookies":{"tpc":"benchmark"}} -> cookieSurvived true. finally block: A already closed, B stop {'success': True} (endTime 23:17:04.627Z), profiles.delete {'success': True}. Printed values match the saved out/result.json and out/06_session_b_visit.json shown at line 126 and the copy at /workspace/home/result.json; a grep -rF of the key over the home dir reported 'no key in home' and saved URLs show token=%3CREDACTED%3E. Authenticity is corroborated by server-side endTime epochs falling inside the local run window, earlier failed probes (401 WebSocket, empty profile list, differing dp-us-west-1/us-east-1 hosts) and real httpbin output - nothing mocked. Note: the preserved snapshot /workspace/home contains only result.json and work/ (docs/auth.md, venv); the hyperbrowser/ code and out/*.json existed at run end per the listing at line 131 but were not captured, so those artifacts were verified from the transcript rather than the snapshot.
Attempt record
- 2:15Answered from saved key: Hyperbrowser API keyno wait
- —First success verified
Attempt artifacts saved.