Benchmark report

Task verified

3m 46s elapsed

The saved attempt met the documented success criterion.

Experience score

96/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current SSInstallSAuthSExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Only minor: docs leave urlRegex matching semantics (full URL vs path) unspecified, and the auth.md OAuth claim flow needs a browser, so the saved key was used instead.

Where the time went

3:46 agent time
  • Research0:32
  • Attempt1:50
  • Verify1:24
3m 46sTotal elapsed
1:10Time to first success
0:00Waited on the owner

Judge reason

Saved code pg-crawl/crawl.py POSTs once to https://api.context.dev/v1/web/crawl with Bearer auth and body {url: https://www.postgresql.org/docs/17/tutorial.html, maxPages: 25, urlRegex: ^https://www[.]postgresql[.]org/docs/17/tutorial, stopAfterMs: 110000} (matches request.json); no client-side link-following or local filtering. response_headers.json shows HTTP 200 with live vendor headers (X-Request-Id 054d1f31-..., X-Credits-Used 24, Cf-Ray, date 2026-11-16). response.json holds 24 results, 24 distinct URLs, all under https://www.postgresql.org/docs/17/tutorial, each with statusCode 200 and real PostgreSQL markdown (e.g. tutorial-window.html -> '3.5. Window Functions'); server metadata numUrls 24, numSkipped 659 confirms the include filter was applied server-side. run_output.txt printed count and URL/title list matches the saved response exactly (programmatic comparison: identical 24 pairs) and titles match the actual PG 17 tutorial pages. Single synchronous job, so no polling applies.

Attempt record

  1. 1:03Answered from saved key: CONTEXT_DEV_API_KEYno wait
  2. —First success verified

Attempt artifacts saved.