Benchmark report
Task verified
9m 43s elapsed
The saved attempt met the documented success criterion.
100/ 100
Agent-Ready- SInstall
- SAuth
- SExecute
- SDocs
- SCurrent
Minimal: WebFetch returned only summarized doc pages, so the agent had to curl the raw .md OpenAPI spec to get exact request/response schemas.
Where the time went
9:44 agent time- Starting0:01
- Research0:49
- Attempt6:51
- Verify2:02
Judge reason
Saved code evidence/ask_context.py POSTs the exact question to the documented answer operation POST https://api.context.dev/v1/web/answers (mode=fast, task, json_format) and writes the raw response; it contains no mock or hard-coded result. evidence/response.json records a live HTTP 200 with server-only artifacts (X-Request-Id b999591c-0b9a-413e-97c2-9d7ea5e9f35e, Cf-Ray a58807ce8b458736-PDX, X-Credits-Used 10 matching the doc's 10-credit fast mode, credits_remaining 4712278), json_content.answer 'READ COMMITTED' and sources/citations ['https://www.postgresql.org/docs/current/sql-set-transaction.html'] — an absolute HTTPS URL returned by the product. stdout.txt and transcript idx 45/51 match the saved response exactly. The fetched spec (evidence/answers-doc-raw.md) shows no beta/preview/request-only marking on /web/answers, so the stop clause does not apply. Auth used a Bearer key obtained via benchmark-request; no key material leaked into artifacts (grep check returned 0).
Attempt record
- 4:28Answered from saved key: CONTEXT_DEV_API_KEYno wait
- —First success verified
Attempt artifacts saved.