Benchmark report

Task verified

4m 14s elapsed

The saved attempt met the documented success criterion.

Experience score

94/ 100

Agent-Ready
  • AInstall
  • SAuth
  • SExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth S, Execute S, Docs A, Current SAInstallSAuthSExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
The PEP 668 pip block forcing a venv retry, plus introspecting the SDK to find client.search after beta.search/search.create guesses failed.

Where the time went

4:14 agent time
  • Research0:12
  • Attempt2:27
  • Verify1:35
4m 14sTotal elapsed
1:15Time to first success
0:00Waited on the owner

Judge reason

Saved code parallel-search/search_excerpts.py calls parallel-web 1.3.7 client.with_raw_response.search(objective='PostgreSQL transaction isolation levels', search_queries=['PostgreSQL transaction isolation levels'], mode='advanced') against the documented POST https://api.parallel.ai/v1/search with x-api-key auth. response_full.json records HTTP 200, search_id search_6e360a0abdf92bcc8ec6a374f2b538c7 (matching the x-request-id response header), and 10 results in the documented shape (url/title/publish_date/excerpts). First result title 'PostgreSQL: Documentation: 18: 13.2. Transaction Isolation', absolute URL https://www.postgresql.org/docs/current/transaction-iso.html, and a 2934-char excerpts[0]; printed run_output.txt and first_result.json match the saved body byte-for-byte. An independent raw curl run (search_curl.sh) returned a distinct live search_id search_4c4c07d3cea2eaf22e3f5b5367544212 with HTTP/2 200, 10 results and the same first URL, confirming a real non-mocked service call. No fabrication, hard-coding or substituted scraping; excerpts field exists so no gap claim was needed.

Attempt record

  1. 0:42Answered from saved key: PARALLEL_API_KEYno wait
  2. —First success verified

Attempt artifacts saved.