The Prompting Company

Benchmark report

Task verified

5m 15s elapsed

The saved attempt met the documented success criterion.

Experience score

95/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current ASInstallSAuthSExecuteADocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
Doc fetch failures - proxy refusals on two Steel doc URLs and a 308 redirect to an unfetchable host - forced a detour through llms.txt and reading node_modules/steel-sdk source.

Where the time went

5:16 agent time
  • Research0:32
  • Attempt3:19
  • Verify1:24
5m 15sTotal elapsed
2:07Time to first success
0:00Waited on the owner

Judge reason

Saved code steel-task/steel_playwright.js uses steel-sdk@0.18.0 client.sessions.create().withResponse() (POST /v1/sessions) and client.sessions.release(id) inside a finally block. Saved evidence/session_create_response.json shows httpStatus 201 with id 90beccc4-e7d6-4634-a795-864d15f0e4c9, status "live" and websocketUrl wss://connect.steel.dev?sessionId=90beccc4-...&token=<REDACTED_SESSION_TOKEN>; Playwright connectOverCDP used that URL with the key only in the documented steel-api-key header (any apiKey query param stripped in code). run_output.txt prints CREATE 201, the same SESSION ID, PAGE TITLE: Example Domain, RELEASE 200, {"success":true,"message":"Session released successfully."}, matching evidence/session_release_response.json and evidence/page_title.txt. Transcript line 60 shows the live unredacted run (real JWT connect token, real debugUrl/projectId) before redaction, so nothing is mocked; an independent re-run (session c105e395-...) reproduced identical results. A grep -rl --fixed-strings for the key across the home dir returned no matches (exit 1), and no saved file contains the API key.

Attempt record

  1. 1:57Answered from saved key: Steel API keyno wait
  2. —First success verified

Attempt artifacts saved.