Benchmark report

Task verified

11m 37s elapsed

The saved attempt met the documented success criterion.

Experience score

96/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current SSInstallSAuthSExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Only a guessed API-reference URL 404ing, plus reconciling the doc note that exclude_domains applies only when include_domains is empty; total elapsed 64s.

Where the time went

11:37 agent time
  • Starting0:25
  • Research1:39
  • Attempt6:53
  • Verify2:40
11m 37sTotal elapsed
1:04Time to first success
0:00Waited on the owner

Judge reason

Saved evidence proves a live authenticated call. request.json shows POST https://api.parallel.ai/v1/search with search_queries ["MVCC vacuum"] and the documented advanced_settings.source_policy {include_domains:[postgresql.org], exclude_domains:[wiki.postgresql.org]} in one request; the fetched Source Policy doc confirms that nesting and that both fields are GA (only /v1beta is legacy). response.json is a real API payload (search_id/session_id search_e78339da7834b529315e8cceae940426, usage sku_search, warnings null, 10 results with publish_date and substantive excerpts) and run_output.txt records HTTP 200. Independent re-parse of response.json: 10 results, every host www.postgresql.org, none wiki.postgresql.org. Code (search_mvcc_vacuum.py) prints all results with no local filtering and no site: operator or domain string in the query. The agent also recorded the documented limitation that exclude_domains applies only when include_domains is empty, which the criteria allow.

Attempt record

  1. 5:19Answered from saved key: Parallel API key (x-api-key)no wait
  2. —First success verified

Attempt artifacts saved.