Benchmark report

Task verified

10m 9s elapsed

The saved attempt met the documented success criterion.

Experience score

66/ 100

Needs Alignment
  • SInstall
  • BAuth
  • AExecute
  • BDocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth B, Execute A, Docs B, Current ASInstallBAuthAExecuteBDocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
Hunting for a routine_id: docs publish none, and both documented discovery paths (CLI, MCP) rejected the API key with 403/401 requiring interactive login.

Where the time went

10:09 agent time
  • Research0:25
  • Attempt7:27
  • Verify2:17
10m 9sTotal elapsed
5:51Time to first success
0:00Waited on the owner

Judge reason

Saved code (code/clay.py + code/enrich_stripe.py) makes real urllib HTTPS calls to Clay's Public API with the documented clay-api-key header. evidence/01-create-search.json shows POST /public/v0/search/query-mode HTTP 200 (search_id=search_0tol8ss5FzM9QmZvc47) with live CloudFront/Request-Id headers; evidence/02-company-results.json shows POST /search/query-mode/{id}/run HTTP 200 returning one record: name 'Stripe', domain 'stripe.com', industry 'Technology, Information and Internet', size '5,001-10,000', linkedin_url .../company/stripe. The body matches the documented PublicApiCompanySearchResult / NextQueryModeCompanyResultsResponse schema in docs/openapi.json (all 12 required fields present), and the company was identified by domain via the documented clay.include_company_identifiers(("stripe.com")) query confirmed in evidence/query-reference.json. evidence/run-output.txt printed values match the saved response exactly. Nothing mocked. The planned Routines enrichment path was unusable because Clay publishes no routine_id listing (GET /public/v0/routines 404; CLI v3 workspaces/tools 403 'Insufficient permissions'; MCP tools/list 401 'You must be logged in' — evidence/03-routine-discovery-blocked.json), so the agent used another officially documented, key-authenticated operation meeting the same criteria rather than guessing an endpoint. No API key, access token or bearer token appears in any saved file or printed output (independently grepped; agent also grepped with the live key value and found no match).

Attempt record

  1. 1:53Answered from saved key: Clay API keyno wait
  2. —First success verified

Attempt artifacts saved.