Benchmark report

Task verified

6m 15s elapsed

The saved attempt met the documented success criterion.

Experience score

95/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current ASInstallSAuthSExecuteADocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
Three MCP/API schema mismatches - edit rejecting price-only, cancel rejecting product_type, and a 502 from edit_preview - each needing a corrective retry.

Where the time went

6:16 agent time
  • Research0:33
  • Attempt3:42
  • Verify2:00
6m 15sTotal elapsed
1:11Time to first success
0:00Waited on the owner

Judge reason

Transcript shows the whole chain run only through mcp__coinbase__* tools (no CLI/HTTP to the exchange; Bash was used only for mkdir, date, a decimal computation and writing evidence). Live BTC-USD spot: precheck orders_list OPEN returned [] (no ctrl0 order to cancel); products_get gave quote_increment 0.01, base_increment 0.00000001, base_min_size 0.00000001, quote_min_size 1; best_bid_ask gave bid 73458.06 at 07:22:35Z. I recomputed independently: 20% below = 58766.44, 21% below = 58031.86, size ceil(15/58766.44) = 0.00025525 (notional 15.0001 USD, within 15-25) - all match. One single orders_create (post_only true, GTC, limit, client_order_id ctrl0lol1794468184, timestamp 1794468184 = 07:23:04Z) returned order_id 6254da5d-535b-4a68-b3b4-95951d15a97e. No second create exists anywhere in the transcript. Lookup used the OPEN orders listing (the server exposes no client-order-id lookup tool; the available tool list has only orders_get/orders_list) showing OPEN, 58766.44, 0.00025525, post_only true. orders_edit returned status EDITED and orders_get confirmed limit_price 58031.86, same order_id (no new identifier). orders_cancel returned success true, orders_get showed status CANCELLED, final orders_list OPEN on BTC-USD was [], and orders_fills returned [] (filled_size 0, number_of_fills 0, total_fees 0). Every saved evidence file is byte-faithful to the raw MCP results in the transcript; no credentials appear in any file or printed output and benchmark-request was never called.

Attempt record

  1. —First success verified

Attempt artifacts saved.