The Prompting Company

Benchmark report

Task verified

5m 56s elapsed

The saved attempt met the documented success criterion.

Experience score

83/ 100

Needs Alignment
  • AInstall
  • AAuth
  • SExecute
  • BDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth A, Execute S, Docs B, Current SAInstallAAuthSExecuteBDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
The hosted API reference was unreachable behind a redirect, so endpoint paths had to be reverse-engineered from the installed SDK source; a Cloudflare UA block cost one extra run.

Where the time went

5:57 agent time
  • Starting0:01
  • Research0:38
  • Attempt3:49
  • Verify1:29
5m 56sTotal elapsed
2:42Time to first success
0:00Waited on the owner

Judge reason

Saved code steel_rest_session.py calls the documented REST API (base https://api.steel.dev, steel-api-key header only, key read from a private file) with Python urllib; no browser automation library appears anywhere. Saved responses show the full lifecycle against the live API: POST /v1/sessions body {"timeout":120000} -> HTTP 201 with id 4e2457af-3cb5-45e6-b86c-ff9e1886c988 and echoed timeout 120000 ms (2 min <= 5 min); GET /v1/sessions/{id} -> 200 status "live"; GET /v1/sessions?status=live -> 200 with that same id in sessions[] (totalCount 1); POST /v1/sessions/{id}/release inside the finally block -> 200 {"success":true,"message":"Session released successfully."}; final GET -> 200 status "released", releaseReason "user_requested". run_stdout.txt and run_summary.json match the saved JSON files exactly, and the transcript shows the actual run (after a first attempt that returned a real Cloudflare 1010 403 until a User-Agent header was added). Docs fetched from docs.steel.dev confirm the steel-api-key header and millisecond timeout units. A grep of all saved files with the real key found no leak; only short-lived connect/player tokens were redacted.

Attempt record

  1. 2:18Answered from saved key: Steel API keyno wait
  2. —First success verified

Attempt artifacts saved.