Benchmark report

Task verified

3m 29s elapsed

The saved attempt met the documented success criterion.

Experience score

98/ 100

Agent-Ready
  • AInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth S, Execute S, Docs S, Current SAInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Minor: the 825 KB marketing HTML at /search-api had to be stripped before the agent located the clean .md OpenAPI reference via llms.txt.

Where the time went

3:29 agent time
  • Research0:38
  • Attempt1:45
  • Verify1:06
3m 29sTotal elapsed
1:10Time to first success
0:00Waited on the owner

Judge reason

Saved code project/search.py POSTs the exact query "PostgreSQL transaction isolation levels" (numResults 10) to the documented endpoint https://api.context.dev/v1/web/search with a Bearer key obtained via benchmark-request. evidence/response_headers.json records HTTP 200 with live-service headers (X-Request-Id 3935578e-4cc5-41b3-a96b-28c489323041, X-Credits-Used 1, X-RateLimit-Remaining 706, Cf-Ray); evidence/response_body.json echoes the exact query, cache_metadata status "miss", credits_consumed 1, same request_id, and 10 results in the documented shape. First result has nonempty title "PostgreSQL Transaction Isolation Levels & MVCC | Mydbops", absolute URL https://www.mydbops.com/blog/postgresql-transaction-isolation-levels-guide, and a nonempty description/excerpt, all matching evidence/run_output.txt and first_result.json. docs/web-search-ref.md (fetched OpenAPI) documents results[].description as "Snippet excerpt from the page", confirming the excerpt field used. Nothing mocked or hard-coded; no scraping or local substitute.

Attempt record

  1. 1:12Answered from saved key: Context.dev API keyno wait
  2. —First success verified

Attempt artifacts saved.