The Prompting Company

Benchmark report

Task verified

6m 59s elapsed

The saved attempt met the documented success criterion.

Experience score

80/ 100

Needs Alignment
  • SInstall
  • BAuth
  • SExecute
  • ADocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth B, Execute S, Docs A, Current ASInstallBAuthSExecuteADocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
Two failed auth attempts because the required COINBASE_ENV=live was absent from `coinbase env --help`; the agent had to grep the binary's strings to find it.

Where the time went

6:59 agent time
  • Starting0:01
  • Research0:43
  • Attempt3:55
  • Verify2:20
6m 59sTotal elapsed
2:38Time to first success
0:00Waited on the owner

Judge reason

Saved CLI outputs prove the full live round trip through the official coinbase CLI. Open-order check printed empty (evidence/11-open-orders-btcusd.txt, plus PENDING/QUEUED/CANCEL_QUEUED/EDIT_QUEUED). Start balances USD 74.644666340325141 / BTC 0 (09-balance-start-showzero.txt). Exactly two orders were created (transcript shows only the buy, the sell and one local --dry-run preview; no curl/SDK/HTTP anywhere): buy 9a816ae6-759a-4913-a8a7-cfc76a0d0da9, client_order_id ctrl0mrt1794467036, market_market_ioc quote_size=20, FILLED, avg 73551.81, fill size 19.8192707226 USD (size_in_quote=true), commission 0.1783734365034, filled_size 0.00026946 BTC; sell 779f38c9-1724-4618-83f3-79333c3c4038 base_size=0.00026946 (= BTC received net of fees, exact at base_increment 0.00000001), FILLED, avg 73518.55, commission 0.178292776347, net 19.632015706653 USD. Fee currency USD is corroborated by the balance debit: 74.644666340325141 - 54.647022181221741 = 19.9976441591034 = quote spent + fee = total_value_after_fees, so no BTC was taken as commission. End balances USD 74.279037887874741 / BTC 0: BTC delta exactly 0 (<0.50 USD) and net USD loss 0.3656 (<1.00). Printed values in 23-roundtrip-summary.txt and the final message match the saved fill JSON. No credential material (PEM, key id, long base64) appears in any saved file or printed output; the wrapper reads secrets from files outside the home dir and never echoes them.

Attempt record

  1. 1:16Answered from saved key: Coinbase Advanced Trade CDP API keyno wait
  2. —First success verified

Attempt artifacts saved.