Benchmark report

Task verified

8m 37s elapsed

The saved attempt met the documented success criterion.

Experience score

90/ 100

Agent-Ready
  • AInstall
  • AAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth A, Execute S, Docs S, Current SAInstallAAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
PEP 668 blocking pip plus the user-site path confusion under python3 -I, and the undocumented org_id requirement on GET/POST /projects.

Where the time went

8:38 agent time
  • Starting0:01
  • Research0:18
  • Attempt6:34
  • Verify1:45
8m 37sTotal elapsed
1:50Time to first success
0:00Waited on the owner

Judge reason

Saved code and live outputs verify the full task against the real Neon management API. code/stage1_create_and_query.py POST /projects -> HTTP 201 created project tpcbench-25c9475983 (id calm-hat-54768124, outputs/01_create_project_response.json), connected with psycopg 3.3.6 using sslmode=verify-full sslrootcert=system, ran CREATE TABLE items (id integer PRIMARY KEY, name text), three bound-parameter INSERTs (INSERT INTO items (id, name) VALUES (%s, %s) with (1,alpha)/(2,beta)/(3,gamma), each 'INSERT 0 1'), and a parameterized read-back SELECT id, name FROM items WHERE id >= %s ORDER BY id with params (2,) printing exactly (2, 'beta') then (3, 'gamma') -- identical in outputs/stage1_console.txt, outputs/02_sql_operations.json and the transcript tool result. No string concatenation anywhere. TLS independently corroborated by psql \conninfo (TLSv1.3, TLS_AES_256_GCM_SHA384) and a refused sslmode=disable attempt (outputs/03_tls_verification.json). Cleanup: DELETE /projects/calm-hat-54768124 -> HTTP 200 with the full project body shown, follow-up GET -> HTTP 404 'project not found' (twice) and the org project list no longer contains tpcbench-25c9475983 (outputs/04_cleanup.json, stage3_console.txt, stage3b_cleanup_recheck.txt); the two other tpcbench- projects were created by concurrent runs (02:48:26 and 02:49:18) and the transcript shows this run issued only one POST /projects. Grep of code/, outputs/ and evidence/ found no password or password-bearing URI -- connection_uri and password fields are [REDACTED] by code/redact.py, and the key was read from the benchmark response file, never printed.

Attempt record

  1. 0:35Answered from saved key: NEON_API_KEYno wait
  2. —First success verified

Attempt artifacts saved.