Benchmark report
Task verified
4m 14s elapsed
The saved attempt met the documented success criterion.
94/ 100
Agent-Ready- AInstall
- SAuth
- SExecute
- ADocs
- SCurrent
The PEP 668 pip block forcing a venv retry, plus introspecting the SDK to find client.search after beta.search/search.create guesses failed.
Where the time went
4:14 agent time- Research0:12
- Attempt2:27
- Verify1:35
Judge reason
Saved code parallel-search/search_excerpts.py calls parallel-web 1.3.7 client.with_raw_response.search(objective='PostgreSQL transaction isolation levels', search_queries=['PostgreSQL transaction isolation levels'], mode='advanced') against the documented POST https://api.parallel.ai/v1/search with x-api-key auth. response_full.json records HTTP 200, search_id search_6e360a0abdf92bcc8ec6a374f2b538c7 (matching the x-request-id response header), and 10 results in the documented shape (url/title/publish_date/excerpts). First result title 'PostgreSQL: Documentation: 18: 13.2. Transaction Isolation', absolute URL https://www.postgresql.org/docs/current/transaction-iso.html, and a 2934-char excerpts[0]; printed run_output.txt and first_result.json match the saved body byte-for-byte. An independent raw curl run (search_curl.sh) returned a distinct live search_id search_4c4c07d3cea2eaf22e3f5b5367544212 with HTTP/2 200, 10 results and the same first URL, confirming a real non-mocked service call. No fabrication, hard-coding or substituted scraping; excerpts field exists so no gap claim was needed.
Attempt record
- 0:42Answered from saved key: PARALLEL_API_KEYno wait
- —First success verified
Attempt artifacts saved.