The Prompting Company

Benchmark report

Task verified

7m 25s elapsed

The saved attempt met the documented success criterion.

Experience score

76/ 100

Needs Alignment
  • SInstall
  • BAuth
  • SExecute
  • BDocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth B, Execute S, Docs B, Current ASInstallBAuthSExecuteBDocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
Auth bootstrap: the documented env vars were insufficient, so the agent reverse-engineered the minified CLI bundle to find COINBASE_ENV=live.

Where the time went

7:26 agent time
  • Starting0:01
  • Research1:08
  • Attempt4:19
  • Verify1:58
7m 25sTotal elapsed
3:01Time to first success
0:00Waited on the owner

Judge reason

Saved commands/outputs show the full cycle ran live through the official `coinbase` CLI (cbx.py only sets COINBASE_KEY_ID/KEY_SECRET/TRANSPORT=brokerage/ENV=live and execs the CLI; no HTTP/SDK/MCP). Pre-check listing was empty (03), BTC-USD SPOT increments 0.01/0.00000001, min 1 USD quote (02), best bid 71427.77 (04). Three separate `orders create` calls returned order ids 4fbf3b86…, 7e94bf2e…, 65026e98… with client ids ctrl0mo1/2/3 + shared suffix 1794518126 (07-09). Prices 57142.21/56427.93/55713.66 are exactly 20/21/22% below the bid rounded down to 0.01; sizes 0.00017501/0.00017722/0.00017949 give notionals 10.0005/10.0002/10.0000 USD (>=10, <11). Listing 10 shows all three OPEN, post_only true, limit_limit_gtc, BUY. One batch request `orders cancel order_ids:=[3 ids]` returned success:true for each (12); a prior attempt with product_type was rejected 400 before cancelling anything. Post-cancel listing is empty (13) and each order reads CANCELLED, filled_size 0 (14). No credential material appears in any saved file or printed output.

Attempt record

  1. 2:07Answered from saved key: Coinbase Advanced Trade CDP API keyno wait
  2. —First success verified

Attempt artifacts saved.