Benchmark report

Task verified

13m 43s elapsed

The saved attempt met the documented success criterion.

Experience score

86/ 100

Agent-Ready
  • SInstall
  • SAuth
  • AExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute A, Docs A, Current SSInstallSAuthAExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
The team default 'require Netlify login' (account site_sso_login=true) made the public URL 401 for 60 polls; fixing it needed an undocumented site-scoped sso_login PATCH.

Where the time went

13:44 agent time
  • Research0:23
  • Attempt10:53
  • Verify2:27
13m 43sTotal elapsed
9:34Time to first success
0:00Waited on the owner

Judge reason

Saved logs and artifacts prove the full cycle against the live Netlify REST API. Site tpcbench-rb-1794012664-690bbe created via POST /api/v1/rick-chen/sites (slug resolved via GET /api/v1/accounts, accounts.json). v1 deployed by file-digest + PUT /deploys/{id}/functions/version?runtime=js (deploy 6aee75fa..., state ready, published_at 00:51:12). Public URL https://tpcbench-rb-1794012664-690bbe.netlify.app/.netlify/functions/version observed {"version":"v1","servedAt":"2026-11-07T00:57:48.864Z"}; after v2 deploy (6aee778d..., ready, published 00:57:53) the same URL served {"version":"v2",...00:57:58.166Z}. Rollback used only the documented POST /api/v1/sites/{id}/deploys/{v1_deploy_id}/restore (HTTP 200, restore_response.json id=6aee75fa..., published_at 00:57:58.313) with no PUT/upload after it in run.log, and the same URL then served {"version":"v1",...00:57:59.159Z} plus four confirmation fetches 00:58:07-00:58:19, all v1 with fresh per-request timestamps. Version is a string literal in code/v1/version.js and code/v2/version.js, servedAt = new Date().toISOString() per request; no env var. Cleanup: DELETE /api/v1/sites/{id} -> 204, post-delete fetch -> 404 'Not Found - Request ID: 01M6SR7SX0XPH5FDF20AE2DZYE', GET /sites?per_page=100 shows zero remaining tpcbench- sites. Grep of all saved files found no bearer token (driver redacts it; only '<REDACTED>' appears).

Attempt record

  1. 1:16Answered from saved key: Netlify personal access tokenno wait
  2. —First success verified

Attempt artifacts saved.