Benchmark report

Task verified

12m 30s elapsed

The saved attempt met the documented success criterion.

Experience score

98/ 100

Agent-Ready
  • AInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth S, Execute S, Docs S, Current SAInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Essentially nothing: first success recorded 51.9s after start, with no failed calls, retries or doc dead ends.

Where the time went

12:30 agent time
  • Starting0:01
  • Research0:41
  • Attempt5:29
  • Verify6:19
12m 30sTotal elapsed
0:52Time to first success
0:00Waited on the owner

Judge reason

Saved evidence proves a live, documented date-filtered Exa search. request.json and search_date_range.py send POST https://api.exa.ai/search with body {query: 'PostgreSQL 17 release', startPublishedDate: '2024-09-01T00:00:00.000Z', endPublishedDate: '2024-10-31T23:59:59.999Z', numResults: 10} authenticated via the documented x-api-key header, key read at runtime from the benchmark-provided file (transcript line 23: EXA_API_KEY_FILE=... python3 -I search_date_range.py). run_stdout.txt shows HTTP 200, requestId=435d4c72635c330ba5c1a70b40257b0d; response.json carries live-response markers (searchTime 3125.5, costDollars {total: 0.007, neural: 0.007}) and 10 results, far above the 3-result minimum. All 10 results carry publishedDate and all fall inside the window (2024-09-05 through 2024-10-17); an independent re-check in verification.txt reports 'out-of-range dated results: 0 []'. The range appears nowhere in the query text and the script prints every result returned without local filtering. Docs fetched (exa.ai/docs/reference/search, exa.ai/auth.md) confirm startPublishedDate/endPublishedDate are standard, non-beta parameters, so the beta/stop clause does not apply. No fabrication or mocking: the request was executed through the real API host with a real key.

Attempt record

  1. 3:36Answered from saved key: Exa API keyno wait
  2. —First success verified

Attempt artifacts saved.