Benchmark report
Task verified
3m 46s elapsed
The saved attempt met the documented success criterion.
100/ 100
Agent-Ready- SInstall
- SAuth
- SExecute
- SDocs
- SCurrent
Only a few minutes of up-front doc fetching (map reference, overview, filter-and-search) before the first and only API call, which succeeded at 81s.
Where the time went
3:46 agent time- Starting0:01
- Research0:21
- Attempt2:02
- Verify1:22
Judge reason
Single live GET https://api.context.dev/v1/web/urls (documented 'Map URLs' URL-discovery endpoint, not Search/Answers) with domain=www.postgresql.org, urlRegex=^https://www\.postgresql\.org/docs/17/, search=tutorial (documented filter per evidence/map-filter-search.md) and maxLinks=50 (documented limit). evidence/map-http-meta.json shows HTTP 200 with live Cloudflare headers, X-Request-Id 4e67f1fd-a6b4-468d-8cb4-83f11210a222 matching the body's request_id, X-Credits-Used 2. evidence/map-response.json holds 24 unique URLs, all under https://www.postgresql.org/docs/17/; my own re-parse confirmed count=24, all-under-prefix=True. project/map_urls.py makes exactly one urllib request, prints len(urls) and urls[:10] in server order with no predicate filtering and no page fetching; evidence/map-run-stdout.txt matches the saved response. Docs snapshots carry no beta/preview/early-access marking.
Attempt record
- 1:14Answered from saved key: CONTEXT_DEV_API_KEYno wait
- —First success verified
Attempt artifacts saved.