Benchmark report

Task verified

10m 8s elapsed

The saved attempt met the documented success criterion.

Experience score

90/ 100

Agent-Ready
  • SInstall
  • SAuth
  • AExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute A, Docs S, Current SSInstallSAuthAExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
The undocumented-at-call-site 422 'Cannot delete persistent branch.' forced a rewrite of cleanup, plus a buffered background script that hid progress for ~2 minutes.

Where the time went

10:08 agent time
  • Research0:18
  • Attempt8:15
  • Verify1:35
10m 8sTotal elapsed
3:27Time to first success
0:00Waited on the owner

Judge reason

Saved code and real API responses support every criterion. POST /v1/projects -> 201 created tpcbench-8fc1a45c7296 (ref zraowsygnnodlljhcflb); items+2 rows seeded via POST /v1/projects/{ref}/database/migrations (200, migration create_items 20261102031635). Branch made through the documented endpoint POST /v1/projects/{ref}/branches -> 201 (branch id 8a44b1dc-27df-47c2-99df-86d3ea33d9ea, project_ref wigzujcqagycpqwsfpmn); outputs/06_branch_list.json shows it alongside main. Branch-only change applied with stock psql over aws-0-us-east-1.pooler.supabase.com:5432, PGSSLMODE=require, \conninfo showing TLSv1.3 (outputs/07_branch_alter.txt: ALTER TABLE / INSERT 0 1). outputs/08_compare.txt shows branch columns id,name,note, has_note_column=t, 3 rows (gamma 'added on branch only') while parent shows id,name, has_note_column=f, 2 rows. Cleanup through the API: first DELETE /v1/branches/{id} returned 422 'Cannot delete persistent branch.', then documented PATCH {persistent:false} -> 200 and DELETE -> 200 {"message":"ok"}; follow-up reads GET /v1/branches/{id} -> 404, branch list shows only default main, GET /v1/projects/wigzujcqagycpqwsfpmn -> 404 'Project not found'. DELETE /v1/projects/zraowsygnnodlljhcflb -> 200, follow-ups GET that ref -> 403 and GET /v1/projects no longer lists either ref. No password or password-bearing connection string in artifacts: sbapi.redact() blanks db_pass/jwt_secret/connection_string, passwords were held in 0600 /tmp files passed via PGPASSWORD and shredded; the only postgresql:// strings carry Supabase's literal [YOUR-PASSWORD] placeholder. Nothing mocked or hard-coded; the only tpcbench project left on the account predates the run (created 03:02:35, before the session began at 03:14) and was correctly left alone.

Attempt record

  1. 0:32Answered from saved key: SUPABASE_ACCESS_TOKENno wait
  2. —First success verified

Attempt artifacts saved.