The Prompting Company

Benchmark report

Task verified

8m 2s elapsed

The saved attempt met the documented success criterion.

Experience score

86/ 100

Agent-Ready
  • AInstall
  • SAuth
  • AExecute
  • SDocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth S, Execute A, Docs S, Current AAInstallSAuthAExecuteSDocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
The global npm install failing on /usr/lib permissions, plus `vercel project rm --yes` being rejected and requiring the --confirm-name form on a second call.

Where the time went

8:03 agent time
  • Research0:24
  • Attempt5:44
  • Verify1:53
8m 2sTotal elapsed
2:52Time to first success
0:00Waited on the owner

Judge reason

Saved evidence supports every criterion. code/v1/api/index.js and code/v2/api/index.js hard-code `const VERSION = 'v1'/'v2'` and compute servedAt with new Date().toISOString() per request (evidence/06 shows two different timestamps). vercel deploy --prod created dpl_2fdTPXrx... (evidence/04) aliased to https://tpcbench-20261107-9e6c4024.vercel.app, which served {"version":"v1",...} (05). After the v2 deploy dpl_Do5Pq6... (07) the same URL served {"version":"v2","servedAt":"2026-11-07T01:19:01.285Z"} (08). The documented Instant Rollback `vercel rollback dpl_2fdTPXrx... --timeout 5m` succeeded in 2s (09) and the same URL then served {"version":"v1","servedAt":"2026-11-07T01:19:17.355Z"} (10). No rebuild: evidence/11 shows exactly two deployments after rollback and targets.production.id == the original v1 deployment id created 01:18:19Z. Cleanup: `vercel project rm ... --confirm-name` deleted the only tpcbench-* project, the tpcbench- filtered project list is [] (13), GET /v9/projects/<proj> returns 404 (15), and the final fetch returned HTTP 404 DEPLOYMENT_NOT_FOUND (16). A token scan over the home dir found no match and no artifact contains a bearer/access token (only VERCEL_TOKEN="$(cat $TOKEN_FILE)" indirection in commands.sh). Account/team resolved via GET /v2/user and /v2/teams.

Attempt record

  1. 1:04Answered from saved key: Vercel API tokenno wait
  2. —First success verified

Attempt artifacts saved.