Benchmark report

Task verified

7m 29s elapsed

The saved attempt met the documented success criterion.

Experience score

79/ 100

Needs Alignment
  • SInstall
  • BAuth
  • SExecute
  • ADocs
  • BCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth B, Execute S, Docs A, Current BSInstallBAuthSExecuteADocsBCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Authentication: documented env vars were not enough, so the agent reverse-engineered the minified CLI bundle to discover COINBASE_ENV=live before any API call worked.

Where the time went

7:29 agent time
  • Research0:51
  • Attempt4:26
  • Verify2:12
7m 29sTotal elapsed
2:04Time to first success
0:00Waited on the owner

Judge reason

Saved CLI transcripts prove the whole chain on live BTC-USD. Increments/min size printed from `coinbase products get` (0.01 / 1e-8 / quote_min_size 1, artifact 03) and best bid 73661.26 from `products best-bid-ask` (05). Pre-flight `orders list status==OPEN` was empty, so no ctrl0 order to cancel (04). Computations check out: 73661.26*0.80=58929.008 -> 58929.00; 15/58929=0.000254543 -> 0.00025455 (notional 15.0004 USD, within 15-25). One post-only GTC limit BUY placed with client_order_id ctrl0lol1794454642 returning order_id 5ba7bca0-9a39-41df-bff8-2185f815704f (09, matching transcript call 26). Lookup via the open-orders listing showed status OPEN, limit_price 58929, base_size 0.00025455 (10); direct lookup by client order ID is documented-absent (`orders get <order_id>` only, HTTP 400 'invalid OrderId', and the list response schema omits client_order_id - artifacts 01/12/13/14), so the permitted listing fallback applies. `orders edit` amended to 58192.39 = floor(73661.26*0.79) (16), confirmed by re-get with order_id unchanged (17). Cancel ran in a trap EXIT cleanup script (scripts/cleanup.sh, 18), final get shows CANCELLED (19), open orders empty (20), `orders fills` empty with filled_size 0 (21). Every exchange call went through the coinbase CLI via a wrapper that only injects env vars and execs the binary - no HTTP, SDK or MCP in any of the 43 tool calls. My own grep of artifacts/, scripts/ and evidence files found no key id or private key material.

Attempt record

  1. 1:38Answered from saved key: Coinbase Advanced Trade API key (CDP) with Trade permissionno wait
  2. —First success verified

Attempt artifacts saved.