Benchmark report

Task verified

4m 58s elapsed

The saved attempt met the documented success criterion.

Experience score

100/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs S, Current SSInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Nothing material: the whole task ran in ~98s; only minor doc ambiguity, as pricing docs say generic 'credits' without naming the lead_credit bucket.

Where the time went

4:59 agent time
  • Research0:24
  • Attempt2:34
  • Verify2:00
4m 58sTotal elapsed
1:39Time to first success
0:00Waited on the owner

Judge reason

Saved code apollo/code/reconcile_credits.py makes exactly the three documented live calls and the run output in the transcript (idx 32/35) shows '[1/3] usage_stats BEFORE -> HTTP 200', '[2/3] organizations/enrich?domain=stripe.com -> HTTP 200', '[3/3] usage_stats AFTER -> HTTP 200'. Saved responses match: 01_usage_before.json and 03_usage_after.json are POST /api/v1/usage_stats/credit_usage_stats at HTTP 200 with lead_credit consumed 2886->2887 and left_over 3734->3733 (all 10 other credit types unchanged), and 02_organization_enrich_stripe.json is GET /api/v1/organizations/enrich?domain=stripe.com at HTTP 200 returning a real 63-field Stripe record (id 5d0a0fbff6512580bf33a120, primary_domain stripe.com) - not mocked. 04_reconciliation.json computes total delta 1 from those saved files and the report states agreement with the documented '1 credit per organization' from docs.apollo.io/reference/organization-enrichment and docs.apollo.io/docs/api-pricing, with usage_stats at 0 credits. Auth is x-api-key only (no bearer/access token exists); the key was read inside the script from the private benchmark file, never printed, and a grep -rqF of the key value across apollo/ returned 'no api key present in artifacts' - my own scan of all artifacts found only $APOLLO_API_KEY / x-api-key variable references, no literal secret.

Attempt record

  1. 1:13Answered from saved key: Apollo.io API keyno wait
  2. —First success verified

Attempt artifacts saved.