Benchmark report

Task verified

6m 0s elapsed

The saved attempt met the documented success criterion.

Experience score

96/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current SSInstallSAuthSExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
The OpenAPI spec URL returned an HTML Scalar docs page instead of JSON, forcing several probes before extracting the embedded spec blob.

Where the time went

6:01 agent time
  • Research0:22
  • Attempt3:57
  • Verify1:42
6m 0sTotal elapsed
2:30Time to first success
0:00Waited on the owner

Judge reason

Saved code and live outputs support every criterion. step2_create.py posted to https://api.supabase.com/v1/projects (stdlib urllib, Bearer token read from a private file) and 03_create_project.json plus transcript node 73 show HTTP 201 with ref qfnlohsahgpoadcwrdzt, name tpcbench-1fb041db, region us-east-1; step3_wait.py polled GET /v1/projects/{ref} to ACTIVE_HEALTHY (04_project_ready.json) and read GET /v1/projects/{ref}/config/database/pooler (05_pooler_config.json -> aws-0-us-east-1.pooler.supabase.com:6543, user postgres.qfnlohsahgpoadcwrdzt). step4_connect.py ran stock psql 16.15 with PGSSLMODE=require against the pooler host; 06_psql_select1.txt records \conninfo 'SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384)', 'select_one 1 (1 row)', server PostgreSQL 17.11, exit_code=0 - a real driver round trip, nothing mocked. step5_delete.py issued DELETE /v1/projects/qfnlohsahgpoadcwrdzt -> HTTP 200 (07_delete_project.json), and two follow-up reads confirm removal: GET /v1/projects/{ref} -> 404 'Resource has been removed' (08) and GET /v1/projects -> only the pre-existing ref vttltanxrmzqekmwfyav (09, 10_cleanup_verdict.json gone=true). My own grep of /workspace/home found no token and no password-bearing connection URI; the only URI present carries the vendor placeholder [YOUR-PASSWORD], and the password was passed to psql via PGPASSWORD and shredded.

Attempt record

  1. 0:37Answered from saved key: SUPABASE_ACCESS_TOKENno wait
  2. —First success verified

Attempt artifacts saved.