Benchmark report

Task verified

23m 47s elapsed

The saved attempt met the documented success criterion.

Experience score

75/ 100

Needs Alignment
  • SInstall
  • SAuth
  • BExecute
  • ADocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute B, Docs A, Current ASInstallSAuthBExecuteADocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
Supavisor pooler behavior: the API-reported transaction port plus warm backends holding stale role OIDs forced port discovery, pool-drain waits and two full reruns.

Where the time went

23:47 agent time
  • Research0:28
  • Attempt21:15
  • Verify2:04
23m 47sTotal elapsed
13:35Time to first success
0:00Waited on the owner

Judge reason

Verified from saved scripts and raw outputs, not prose. work/scripts/05_create_role.sh calls the documented POST /v1/projects/wrtpqenldcujonpbaokq/cli/login-role {"read_only":true} -> 201 {role:cli_login_supabase_read_only_user, ttl_seconds:300} (05_create_login_role_response.redacted.json); 05b scoped it via POST /database/query and 05b_role_grants.txt shows exactly one grant (items, SELECT). credential_lifecycle_final.log (transcript idx 300, 03:25-03:27) shows psql over the required pooler host aws-0-us-east-1.pooler.supabase.com:5432 sslmode=require with \conninfo 'SSL connection (TLSv1.3, TLS_AES_256_GCM_SHA384)', current_user=cli_login_supabase_read_only_user, SELECT returning 3 rows (exit 0), INSERT 'ERROR: permission denied for table items' (exit 1), UPDATE/DELETE refused, privilege map t/f/f/f, row_count still 3. Revocation: DELETE /cli/login-role -> 200 {"message":"ok"} (07b), follow-up pg_roles read matching_roles=0 (07c), and reconnect with the same credential fails through 'invalid role OID' then 'FATAL: password authentication failed' (psql exit 2). Cleanup: DELETE /v1/projects/wrtpqenldcujonpbaokq -> 200 with body, GET the ref -> 404 {"message":"Project not found"}, GET /v1/projects -> 0 tpcbench-* remaining (08a/08b/08c). Password kept in /tmp/bench-private/role_password (0600, shredded; dir empty at 03:28) and passed via PGPASSWORD; the agent's own grep over all artifacts found no password or password-bearing connection string, and the pooler connection_string fields were redacted before saving. Nothing mocked or hard-coded: every artifact is a tee'd live curl/psql response reproduced in the transcript.

Attempt record

  1. 0:48Answered from saved key: SUPABASE_ACCESS_TOKENno wait
  2. —First success verified

Attempt artifacts saved.