Benchmark report
Task verified
6m 25s elapsed
The saved attempt met the documented success criterion.
89/ 100
Agent-Ready- SInstall
- AAuth
- SExecute
- ADocs
- SCurrent
WebFetch returned summaries instead of raw docs and two guessed doc URLs 404'd, forcing a fallback to curl of llms.txt/llms-full.txt to find CDP and auth details.
Where the time went
6:26 agent time- Research0:25
- Attempt4:15
- Verify1:46
Judge reason
Saved code (work/app/persist-cookie.mjs, identical copy work/code-persist-cookie.mjs) and per-call responses show the full documented flow executed live against api.onkernel.com via @onkernel/sdk 0.139.0 + playwright-core: profiles.create -> HTTP 201, profile id bo8ujli1ti4mbulci2owqrrs (01-profile-create.json); session A browsers.create with profile{name,save_changes:true} -> HTTP 200, session_id i7caqkjagaqkuj5987uqsa1h, profile_save_changes true (02); Playwright chromium.connectOverCDP(cdp_ws_url) loaded httpbin.org/cookies/set?tpc=benchmark -> 200, page body {"cookies":{"tpc":"benchmark"}} (03); session A terminated with browsers.deleteByID -> 204 before B was created, and profiles.retrieve confirmed the documented save (last_used_at 23:24:13.389915Z, 04/05); session B created at 23:24:17 with the same profile -> HTTP 200, distinct session_id v2lupvakaj4193teg5o9pe2s (06); CDP connect to B loaded httpbin.org/cookies -> 200 returning {"cookies":{"tpc":"benchmark"}} plus context cookie tpc=benchmark domain httpbin.org (07). The finally block deleted session B (204) and the profile (204) and re-retrieve returned 404 profile not found (08). Docs saved under work/docs confirm these are the documented operations (profiles.md: 'Deleting the Kernel browser persists its state when save_changes is enabled'; OpenAPI securitySchemes bearerAuth; DELETE /profiles/{id_or_name}, DELETE /browsers/{id_or_name}). Printed console output (run-console.log, matching the live tool result in the transcript) matches the saved JSON responses exactly; cdp jwt query params are redacted and a byte-level scan of the whole home dir for the key value returned 'leaks: []' — the key went only in the SDK's Authorization: Bearer header, never a URL. No mocking or hard-coded responses.
Attempt record
- 2:15Answered from saved key: KERNEL_API_KEYno wait
- —First success verified
Attempt artifacts saved.