Benchmark report

Task not completed

6m 39s elapsed

The saved evidence did not demonstrate the documented success criterion.

Experience score

60/ 100

Needs Alignment
  • SInstall
  • SAuth
  • CExecute
  • BDocs
  • BCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute C, Docs B, Current BSInstallSAuthCExecuteBDocsBCurrent
Agent profile · this run
Judge's note · what slowed the agent most
kraken_order_cancel_batch returning EOrder:Invalid order for four different valid-ID encodings against three confirmed-open orders, consuming ~70s of probing before fallback.

Where the time went

6:40 agent time
  • Research0:27
  • Attempt4:19
  • Verify1:53
6m 39sTotal elapsed
—Time to first success
0:00Waited on the owner

Judge reason

Setup and placement are fully verified from saved MCP tool calls/results: kraken_pairs gave tick_size 0.1, lot_decimals 8, ordermin 0.00006; kraken_ticker gave best bid 73482.7; kraken_server_time gave 1794468884; three separate kraken_order_buy calls created post-only GTC limit buys at 58786.1/58051.3/57316.5 (-20/-21/-22%, each ~$10.00 notional) with cl_ord_ids ctrl0mo1/2/3 + 1794468884 and txids ODMH4K-IELSE-D3I4LS, O6GNNZ-EBD6O-6XTRWU, OPTTWK-MXR5Y-EQJEZU, and kraken_open_orders showed all three open (oflags post,fciq; gtc; volume_executed 0). The core criterion failed: mcp__kraken__kraken_order_cancel_batch returned {"error":"api","message":"EOrder:Invalid order"} on all four attempts (txids; cl_ord_ids; both; comma-joined) at transcript lines 50/56/70/115, with open_orders and query_orders between attempts confirming the txids were still open and valid, so no batch-cancel response confirms any cancellation. 'Not supported' does not apply because Kraken documents CancelOrderBatch. The three orders were cancelled only by three individual kraken_order_cancel calls (count:1 each); final kraken_open_orders is empty and query_orders shows all three status=canceled with volume_executed 0, so cleanup and no-fill are satisfied, but the single-batch-cancel requirement is not. No 502/503 credential-proxy error occurred and no credentials appear in evidence; every exchange call went through MCP tools.

Attempt record

  1. 4:47Success criterion not demonstrated

Attempt artifacts saved.