Benchmark report

Task verified

7m 25s elapsed

The saved attempt met the documented success criterion.

Experience score

95/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current ASInstallSAuthSExecuteADocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
Undocumented unstable result ordering: default relevance paging repeated an id across pages, forcing two re-runs plus a probe script before pages were disjoint.

Where the time went

7:26 agent time
  • Research0:35
  • Attempt4:41
  • Verify2:09
7m 25sTotal elapsed
3:03Time to first success
0:00Waited on the owner

Judge reason

Saved code (code/apollo_people_search.py) makes two separate authenticated POSTs to Apollo's documented People API Search (https://api.apollo.io/api/v1/mixed_people/api_search, docs.apollo.io/reference/people-api-search) with person_titles[]=Software Engineer, q_organization_domains_list[]=stripe.com, per_page=5 and page=1 then page=2, using the x-api-key header read from the benchmark-supplied private file. output/run_output.txt shows auth/health HTTP 200 {"healthy":true,"is_logged_in":true} and both searches HTTP 200 with 5 people each, total_entries=2651. Independent re-check of output/page1_redacted.json and page2_redacted.json confirms 5 ids per page, zero id overlap, and that printed ids/titles/total_entries match the saved files exactly. Only first_name and last_name_obfuscated are replaced with "[redacted]" (10 per file = 5 people x 2 fields); ids, titles, last_refreshed_at, has_* flags, organization.name and total_entries are unchanged; this endpoint returns no emails by design and a regex scan of /workspace/home finds no email address, API key, or bearer/access token in any saved file or printed output. Responses are demonstrably live, not mocked: total_entries changed between runs (2826 default vs 2651 with include_similar_titles=false) and page composition drifted across trials. The response body carries no page/per_page/total_pages, which the report states explicitly rather than inventing.

Attempt record

  1. 1:07Answered from saved key: Apollo.io API keyno wait
  2. —First success verified

Attempt artifacts saved.