Benchmark report

Task verified

5m 17s elapsed

The saved attempt met the documented success criterion.

Experience score

100/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs S, Current SSInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Almost nothing: one self-inflicted f-string syntax fix to its own script before the single successful run; no product-side friction.

Where the time went

5:18 agent time
  • Research0:34
  • Attempt3:09
  • Verify1:34
5m 17sTotal elapsed
1:39Time to first success
0:00Waited on the owner

Judge reason

Saved code apollo/code/reconcile_credits.py calls POST /api/v1/usage_stats/credit_usage_stats, exactly one GET /api/v1/organizations/enrich?domain=stripe.com, then the usage endpoint again, all with the documented x-api-key header. Saved responses 01/02/03 each record http_status 200; 02 contains a real Stripe org (id 5d0a0fbff6512580bf33a120, 100KB of live fields), so nothing is mocked. lead_credit.consumed goes 32 -> 33 and left_over 63 -> 62 between the readings; REPORT.md prints both readings, the +1 delta and the documented price (1 credit per organization, docs.apollo.io/reference/organization-enrichment; usage op 0 credits, docs.apollo.io/reference/view-credit-usage-stats) and states they agree exactly, noting the 10s wait inside the documented 'update within seconds' window. The key was read only from APOLLO_KEY_FILE; a grep of outputs/ and code/ for the key value and for token fields found none, and my own scan of all artifacts found only Apollo object IDs, no token or key material. benchmark-first-success.json points to apollo/outputs/EVIDENCE_first_success.txt, which was written from the completed run (three HTTP 200s, both readings, delta table) before /usr/local/bin/benchmark-success ran at transcript step 44, so the planned operation had already succeeded when marked.

Attempt record

  1. 1:06Answered from saved key: Apollo.io API keyno wait
  2. —First success verified

Attempt artifacts saved.