Benchmark report

Task verified

5m 6s elapsed

The saved attempt met the documented success criterion.

Experience score

99/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • SDocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs S, Current ASInstallSAuthSExecuteSDocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
PEP 668 blocked a direct pip install, forcing a venv; otherwise the only wait was polling project status, which was already ACTIVE_HEALTHY.

Where the time went

5:06 agent time
  • Research0:24
  • Attempt3:07
  • Verify1:35
5m 6sTotal elapsed
1:56Time to first success
0:00Waited on the owner

Judge reason

Saved evidence supports every criterion. work/code/run_sql.py connects with psycopg 3.3.9 to the Supavisor pooler host aws-0-us-east-1.pooler.supabase.com:5432 with sslmode=require (redacted DSN in out/07-sql-detail.json), creates 'items (id integer primary key, name text)', inserts (1,alpha)(2,beta)(3,gamma) via cur.execute("insert into items (id, name) values (%s, %s)", r) bound params (three INSERT 0 1 statusmessages), and selects with "where id >= %s order by id" bound to 2. out/06-sql-stdout.txt prints exactly (2, 'beta') then (3, 'gamma'), matching the rows recorded in out/07-sql-detail.json; transcript idx 59-60 shows that run executing live (exit=0, server_version PostgreSQL 17.11). Project tpcbench-95luyjcr (ref scolxudwcqrmjmsjvdvo) was created live via POST /v1/projects (201, out/03-create-project.json) and deleted via DELETE /v1/projects/{ref} (200, out/09-delete-project.json); follow-up GET returns 404 "Resource has been removed" (out/10) and the project is absent from GET /v1/projects (out/11). My own grep of /workspace/home found no token and no password-bearing DSN - only ***REDACTED*** forms and the API's own [YOUR-PASSWORD] template. pg_stat_ssl ssl=f behind the pooler is explicitly allowed. Nothing mocked or hard-coded.

Attempt record

  1. 1:03Answered from saved key: SUPABASE_ACCESS_TOKENno wait
  2. —First success verified

Attempt artifacts saved.