Benchmark report

Task not completed

12m 33s elapsed

The saved evidence did not demonstrate the documented success criterion.

Experience score

60/ 100

Needs Alignment
  • SInstall
  • SAuth
  • CExecute
  • BDocs
  • BCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute C, Docs B, Current BSInstallSAuthCExecuteBDocsBCurrent
Agent profile · this run
Judge's note · what slowed the agent most
The CLI's cancel-batch returning EOrder:Invalid order for every documented input form, costing three full runs plus eight diagnostic probes with live orders.

Where the time went

12:34 agent time
  • Research0:27
  • Attempt9:41
  • Verify2:25
12m 33sTotal elapsed
—Time to first success
0:00Waited on the owner

Judge reason

Setup and placement verified: the official kraken CLI authenticated via env vars (artifacts/run3 'auth test' -> {"status":"success","source":"environment variable"}), read pair params from the API (tick 0.1, lot 1e-8, ordermin 6e-05) and best bid 73666.9 (run3/21_ticker.json), and placed exactly three live post-only GTC limit buys with one request each at 58933.5/58196.8/57460.1 (20/21/22% below bid, each ~10.0004 USD) under cl ord IDs ctrl0mo1/2/3 + suffix 1794457453; run3/50_open_orders_before_cancel.json shows all three open. The required single batch-cancel failed: 'kraken order cancel-batch <3 txids>' returned {"error":"api","message":"EOrder:Invalid order"} (run3/60_cancel_batch_rest.json), as did every other documented form across runs and probes (--cl-ord-ids in run1, txids in run2, single valid txid d2, comma form d3, userref u_batch_userref), and the ws batch-cancel fallback died on connection reset (run3/61_cancel_batch_ws.err). run3/71_after_summary.txt shows all three still open after the batch attempt; they were only removed one at a time in cleanup (91_cleanup_cancel_1..3.json). The task's criteria require one batch-cancel request confirming each cancellation, and the Not-supported exemption does not apply because the CLI documents cancel-batch (final/82_cli_cancel_batch_help.txt) and it hits POST /0/private/CancelOrderBatch (final/84_cancel_batch_endpoint_proof.json). Final state is clean (final/80_open_orders_final.json {"open":{}}, all closed with vol_exec 0, fee 0) and no credential appears in any saved file; this is a product/CLI defect, not an external prerequisite.

Attempt record

  1. 1:35Answered from saved key: Kraken Spot API credentialsno wait
  2. 10:09Success criterion not demonstrated

Attempt artifacts saved.