Benchmark report
Task verified
3m 29s elapsed
The saved attempt met the documented success criterion.
98/ 100
Agent-Ready- AInstall
- SAuth
- SExecute
- SDocs
- SCurrent
Minor: the 825 KB marketing HTML at /search-api had to be stripped before the agent located the clean .md OpenAPI reference via llms.txt.
Where the time went
3:29 agent time- Research0:38
- Attempt1:45
- Verify1:06
Judge reason
Saved code project/search.py POSTs the exact query "PostgreSQL transaction isolation levels" (numResults 10) to the documented endpoint https://api.context.dev/v1/web/search with a Bearer key obtained via benchmark-request. evidence/response_headers.json records HTTP 200 with live-service headers (X-Request-Id 3935578e-4cc5-41b3-a96b-28c489323041, X-Credits-Used 1, X-RateLimit-Remaining 706, Cf-Ray); evidence/response_body.json echoes the exact query, cache_metadata status "miss", credits_consumed 1, same request_id, and 10 results in the documented shape. First result has nonempty title "PostgreSQL Transaction Isolation Levels & MVCC | Mydbops", absolute URL https://www.mydbops.com/blog/postgresql-transaction-isolation-levels-guide, and a nonempty description/excerpt, all matching evidence/run_output.txt and first_result.json. docs/web-search-ref.md (fetched OpenAPI) documents results[].description as "Snippet excerpt from the page", confirming the excerpt field used. Nothing mocked or hard-coded; no scraping or local substitute.
Attempt record
- 1:12Answered from saved key: Context.dev API keyno wait
- —First success verified
Attempt artifacts saved.