Benchmark report
Task verified
16m 17s elapsed
The saved attempt met the documented success criterion.
68/ 100
Needs Alignment- SInstall
- BAuth
- AExecute
- ADocs
- CCurrent
Steel's CDP socket rejects the API key in any header (only ?apiKey= or connect token), costing ~5 min of probing; then docs' nonexistent profiles.delete cost another ~4 min.
Where the time went
16:18 agent time- Research0:32
- Attempt12:45
- Verify3:01
Judge reason
Saved code (work/steel_profile_cookie.mjs) and its real run output prove the whole flow. Session A was created with client.sessions.create({persistProfile:true}) which minted profile e90c3e56-7258-49dc-a9c0-83e1e6ae9ebd (the documented profile-creation path per docs/profiles-overview.md), id printed in run.log and saved in evidence/01. Playwright connectOverCDP to session A's websocketUrl loaded httpbin.org/cookies/set?tpc=benchmark (evidence/03, HTTP 200, body {"cookies":{"tpc":"benchmark"}}). sessions.release(A) returned {"success":true,...} (evidence/04) and the profile was polled UPLOADING -> READY (evidence/05) BEFORE session B was created at 23:37:17 (evidence/06, id ba43e369-..., distinct, same profileId). Session B over CDP returned {"cookies":{"tpc":"benchmark"}} from httpbin.org/cookies (evidence/07) - the cookie survived. finally block released both sessions and GET /v1/sessions/{id} shows both "released" (evidence/08). API key sent only in the steel-api-key header; the websocketUrl carries only Steel's own session connect token, which is redacted in every artifact; my own grep of /workspace/home found no key material and the agent's in-run grep -rlF over the home dir printed "no api key in any saved file". Printed run.log values match the saved JSON exactly. One clause could not be met: profile deletion. docs.steel.dev/cookbook/profiles names client.profiles.delete(id), but the call issued from the finally block returned HTTP 421 "No route defined for DELETE /v1/profiles/{id}" (evidence/08), and probes show CORS allow-methods GET,HEAD,PATCH,OPTIONS for /v1/profiles/{id} (evidence/09). I independently confirmed this: steel-sdk 0.18.0 (npm latest) src/resources/profiles.ts exposes only create, update, list, get - no delete. That is a product/doc defect, not an agent shortfall; the documented call was still issued in finally and its exact response saved, so the demonstrated core result stands.
Attempt record
- 3:07Answered from saved key: STEEL_API_KEYno wait
- —First success verified
Attempt artifacts saved.