Benchmark report

Task verified

13m 1s elapsed

The saved attempt met the documented success criterion.

Experience score

95/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current ASInstallSAuthSExecuteADocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
One failed build from Vercel's default nodeVersion 26.x being rejected by the pinned @vercel/node builder, costing a diagnose-and-redeploy cycle.

Where the time went

13:01 agent time
  • Starting0:01
  • Research0:28
  • Attempt9:19
  • Verify3:13
13m 1sTotal elapsed
4:27Time to first success
0:00Waited on the owner

Judge reason

Saved evidence supports every criterion. TPC_BENCH_SECRET was stored via the documented Vercel mechanism POST /v10/projects/{id}/env (outputs/05-add-env-response.txt: status=201, type=sensitive, visibility=secret, value redacted by API), on project tpcbench-d3008c48 created via POST /v11/projects (team resolved via GET /v2/teams). A production deployment (POST /v13/deployments, dpl_9vdXi4kBy2DRrhEUQCFK6dzNSdCg, readyState READY, alias tpcbench-d3008c48-tpc-cao.vercel.app) served HTTP 200 (12-endpoint-headers.txt: HTTP/2 200, server: Vercel, x-vercel-cache: MISS) returning {"sha256":"0f16de1c...4ccf0","source":"process.env.TPC_BENCH_SECRET"}; 12-verify.txt prints LOCAL and REMOTE hashes both 0f16de1c...4ccf0, MATCH=True, with the local hash recomputed in 06-verify.sh from /tmp/private/tpc_secret (generated locally via secrets.token_hex(32), never printed). Deployed source fetched back from Vercel (GET /v6 and /v7 files) and build events contain no secret; the base64 payload in deploy_body.json decodes to api/hash.js with no secret literal; the agent's own leak scan printed 'leak hits: NONE'. Runtime-logs stream returned 502 through the sandbox proxy and the gap was documented, with deployment events captured instead. Cleanup: DELETE /v9/projects -> 204, project lookup 404, tpcbench-* search returned [], and the final fetch printed HTTP/2 404 with x-vercel-error: DEPLOYMENT_NOT_FOUND (an initial post-delete fetch still returned 200, which the agent honestly recorded and then polled to 404). No saved file or output contains an access or bearer token (only resource IDs and a docs placeholder).

Attempt record

  1. 0:57Answered from saved key: Vercel API access tokenno wait
  2. —First success verified

Attempt artifacts saved.