Benchmark report

Task verified

12m 44s elapsed

The saved attempt met the documented success criterion.

Experience score

95/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current ASInstallSAuthSExecuteADocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
The deprecated client_order_id lookup returning HTTP 400, forcing a fallback to filtering the open-orders listing to find the order.

Where the time went

12:45 agent time
  • Starting0:01
  • Research0:25
  • Attempt9:14
  • Verify3:05
12m 44sTotal elapsed
6:27Time to first success
0:00Waited on the owner

Judge reason

Verified from raw saved HTTP responses (genuine Cloudflare/Cf-Ray/Trace-Id headers, api.coinbase.com production host, no sandbox). Step 0: 01-step0 listing returned orders:[] and the run printed 'open orders on pair: 0 | with ctrl0 client order id: 0'. Step 1: 02-step1-get-product gave price_increment 0.01, base_increment 0.00000001, base_min_size 0.00000001, quote_min_size 1; 03-step1-best-bid-ask gave best bid 73626.08. I recomputed independently: floor(73626.08*0.80/0.01)=58900.86 and ceil(15/58900.86/1e-8)=0.00025467 (notional 15.00028 USD), matching the POST /api/v3/brokerage/orders body in 04-step2-create-order (HTTP 200, success true, limit_limit_gtc post_only true, client_order_id ctrl0lol1794446419, order_id 2b7675ad-1ad1-48ec-940d-cc24e4f98e52). Exactly one create-order call exists in the artifacts and the transcript; run_task.py ran once and only one order UUID and one ctrl0lol id appear anywhere. Lookup: the documented-but-deprecated client_order_id lookup returned HTTP 400 INVALID_ARGUMENT (05), so the open-orders listing filtered on client_order_id was used (06) showing status OPEN, limit_price 58900.86, base_size 0.00025467, post_only true, GOOD_UNTIL_CANCELLED. Amend: POST /orders/edit price 58164.60 returned HTTP 200 success true (07) and the retrieval (08/27) shows limit_price 58164.6 with edit_history replace_accept_timestamp and order_id unchanged; 58164.60 equals floor(73626.08*0.79/0.01) exactly. Cleanup ran in run_task.py's finally block: batch_cancel HTTP 200 success true (28), retrieval 29 shows status CANCELLED, cancel_message 'User requested cancel', filled_size 0, number_of_fills 0, average_filled_price 0, total_fees 0; 30 and responses/final-check/01 show no open orders on BTC-USD, and final-check/02 shows USD 75 with hold 0 and BTC 0, confirming no fill. Code is hand-written requests + a manually built ES256 JWS (code/cb_client.py), no vendor SDK/CLI/MCP/exchange library; credentials are read from files via env paths, sent only in the Authorization header (recorded as '<JWT redacted>'), and my own scan of all artifacts found no PEM, key name, or JWT material (the only PEM markers in the transcript are doc placeholders).

Attempt record

  1. 3:10Answered from saved key: Coinbase Advanced Trade CDP API keyno wait
  2. —First success verified

Attempt artifacts saved.