Benchmark report

Task verified

11m 32s elapsed

The saved attempt met the documented success criterion.

Experience score

95/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current ASInstallSAuthSExecuteADocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
The undocumented org_id requirement on POST /projects (400, pointing at the console UI) and the ~5-7 min lag before the deleted project's /sql endpoint stopped answering.

Where the time went

11:33 agent time
  • Research0:24
  • Attempt9:06
  • Verify2:03
11m 32sTotal elapsed
3:13Time to first success
0:00Waited on the owner

Judge reason

Verified from saved artifacts, not prose. work/code/01_create_and_query.mjs imports only neon() from @neondatabase/serverless 1.2.0 (Pool/Client never imported) and runs CREATE TABLE items, INSERT of 2 rows RETURNING, and SELECT. work/out/02-sql-over-http-evidence.json logs three POSTs to https://api.c-8.us-east-2.aws.neon.tech/sql each HTTP 200, all_fetch_urls_are_https true, postgres_tcp_connections (5432) 0, websocket_attempts 0; the printed rows in 01-run.log ([{id:1,widget,7},{id:2,gadget,42}]) match the saved response exactly. A reproduction run (out/run2) shows the same with the fixed socket hook: the only TCP connects are port 8080 CONNECT tunnels to the environment's egress proxy. 04-control.log proves the detector is not a no-op (records a 127.0.0.1:5432 connect and throws on WebSocket). Project tpcbench-ed4ed5cc94 was created via POST https://console.neon.tech/api/v2/projects (201) and deleted via DELETE /projects/hidden-base-77969272 (200, response saved); follow-up GET returned 404 'project not found' and a list of 46 org projects contained no tpcbench- entries; run2's tpcbench-59f535143e was cleaned up identically. Secrets: key requested via benchmark-request and passed only as a file path read inside lib.mjs; connection_uri and password fields are [REDACTED] in every saved response, and a grep across work/ finds credential-bearing URLs only in the vendor docs' [password] placeholders. Nothing mocked or hard-coded.

Attempt record

  1. 0:43Answered from saved key: NEON_API_KEYno wait
  2. —First success verified

Attempt artifacts saved.