Benchmark report

Task verified

9m 9s elapsed

The saved attempt met the documented success criterion.

Experience score

91/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • BDocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs B, Current ASInstallSAuthSExecuteBDocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
No published list of valid organization_industries values (404 page, blocked KB article), so ~3 of 5 minutes went to probe scripts proving 'computer software' is the right value.

Where the time went

9:10 agent time
  • Starting0:01
  • Research0:20
  • Attempt5:52
  • Verify2:57
9m 9sTotal elapsed
4:37Time to first success
0:00Waited on the owner

Judge reason

Verified from saved artifacts, not prose. code/final_company_search.py calls POST https://api.apollo.io/api/v1/mixed_companies/search with x-api-key read from a private file, sending only server-side filters organization_industries[]=computer software, organization_locations[]=germany, organization_num_employees_ranges[]=201,500, page=1, per_page=10 — all documented on the fetched Organization Search page (transcript line 11 lists organization_industries[] under 'Industry'). The transcript (line 156) shows the live run: HTTP 200, pagination {page:1, per_page:10, total_entries:362, total_pages:37}, 10 organizations; outputs/final_search_response.json matches byte-for-byte and has the documented shape (breadcrumbs, pagination, organizations, model_ids). No post-filtering: the code prints every returned org. The industry filter is proven server-side by the agent's own probes — germany+201-500 alone returns 3,629 and adding organization_industries[]=computer software returns 362 (step6_filter_probe_summary.json), while 'software development' and a bogus control both return total_entries=0 (step7_summary.json), confirming exact taxonomy matching and that 'computer software' is Apollo's name for the software-development industry (corroborated by industries[] arrays from Organization Enrichment for SAP, Celonis, Personio, JetBrains, GitLab). The search response carries no industry/headcount/country fields (verified: org keys are id,name,website_url,primary_domain,sic/naics,revenue,...), so the agent read them per company from the documented GET /api/v1/organizations/enrich. I independently re-parsed all 10 final_enrich_*.json files: every company has country=Germany, estimated_num_employees in 201-500 (240-480), and 'computer software' in industries[] — exactly matching final_console_output.txt and final_printed_companies.json. No credential leakage: a grep of all saved files found the key nowhere; only the words 'access_token'/'bearer' appear in the redaction helper's code and prose, and the key was passed only by file path via APOLLO_KEY_FILE.

Attempt record

  1. 1:27Answered from saved key: Apollo.io API keyno wait
  2. —First success verified

Attempt artifacts saved.