Benchmark report

Task verified

14m 13s elapsed

The saved attempt met the documented success criterion.

Experience score

85/ 100

Agent-Ready
  • SInstall
  • SAuth
  • AExecute
  • ADocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute A, Docs A, Current ASInstallSAuthAExecuteADocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
Team default site SSO made the production URL 401 until PATCH sso_login=false, plus rotating the secret after the dev-context value came back in plaintext.

Where the time went

14:14 agent time
  • Research0:17
  • Attempt11:19
  • Verify2:37
14m 13sTotal elapsed
7:16Time to first success
0:00Waited on the owner

Judge reason

Saved evidence supports every criterion. Secret stored via documented Netlify createEnvVars API (outputs/07-createEnvVars-response.json: HTTP 201, is_secret=true, values masked ****919e) on site tpcbench-b65bde06 created through POST /{account_id}/sites (05-create-site-response.json). Deployed with 'netlify deploy --prod --no-build' (08-netlify-deploy.log, deploy 6aee6440... state ready, 1 function). GET https://tpcbench-b65bde06.netlify.app/hash returned HTTP 200 (09-endpoint-headers.txt, server: Netlify) with {"sha256":"36dd1476...1e4abef"}; 12-hash-comparison.txt shows local SHA-256 computed independently from /tmp/bench-priv/secret.txt equals it (MATCH: true). Deployed source site/netlify/functions/hash.mjs reads process.env.TPC_BENCH_SECRET at runtime and logs nothing - no hard-coded value; grep over all saved artifacts finds no 64-hex string other than the digest and no bearer token. Netlify function-log retrieval was blocked by the sandbox proxy (18-analytics-host-probe.txt: CONNECT tunnel failed 403), so no platform log was captured; the gap is documented with evidence. Cleanup: env var DELETE 204, site DELETE 204, GET /sites/{id} 404, GET /sites -> [] (19/20), and final fetch returned HTTP 404 Netlify Not Found with no sha256 field (22-final-fetch.log). Account-level sso_login default left unchanged (02 vs 14 accounts files identical).

Attempt record

  1. 0:44Answered from saved key: NETLIFY_AUTH_TOKENno wait
  2. —First success verified

Attempt artifacts saved.