Benchmark report

Task verified

4m 46s elapsed

The saved attempt met the documented success criterion.

Experience score

100/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs S, Current SSInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Nothing material: 50-line transcript, one doc fetch, one key request, one script run that succeeded first try in under 90 seconds.

Where the time went

4:47 agent time
  • Research0:32
  • Attempt2:25
  • Verify1:50
4m 46sTotal elapsed
1:29Time to first success
0:00Waited on the owner

Judge reason

Verified end-to-end from saved evidence. code/apollo_people_search.py calls the documented Apollo operation POST https://api.apollo.io/api/v1/mixed_people/api_search (doc page fetched in-transcript at docs.apollo.io/reference/people-api-search) with person_titles[]=Software Engineer, q_organization_domains_list[]=stripe.com and documented page/per_page pagination, auth via x-api-key read from a private file (never printed). The single run (transcript idx 25-26) printed 'page=1 per_page=5 HTTP 200' and 'page=2 per_page=5 HTTP 200', total_entries 2827, 5 people each, 'overlapping ids ... none'. My own parse of outputs/page1_redacted.json and page2_redacted.json confirms http_status 200, exactly 5 people per page, total_entries 2827, all titles 'Software Engineer', zero id overlap, and ids/titles identical to console_output.txt and summary.json. Every person record has first_name and last_name_obfuscated == '[redacted]'; an email regex over all saved files returns no matches, and no token-like string or bearer/access_token value appears in any artifact (organization.name 'Stripe' is a company, not a person). Raw responses were redacted in memory before writing, so no unredacted response was stored. Responses are real API data (varied last_refreshed_at timestamps, per-record has_direct_phone values), not mocked.

Attempt record

  1. 1:12Answered from saved key: Apollo.io API keyno wait
  2. —First success verified

Attempt artifacts saved.