Benchmark report

Task verified

5m 13s elapsed

The saved attempt met the documented success criterion.

Experience score

98/ 100

Agent-Ready
  • AInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth S, Execute S, Docs S, Current SAInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Only the post-hoc title spot-check: direct curl to postgresql.org was proxy-blocked (exit 56), so titles were confirmed from the crawl's own markdown h1 instead.

Where the time went

5:13 agent time
  • Research0:17
  • Attempt3:30
  • Verify1:26
5m 13sTotal elapsed
1:57Time to first success
0:00Waited on the owner

Judge reason

Saved code (firecrawl-crawl/crawl_pg_tutorial.py, byte-identical copy in evidence) issues one product-side POST https://api.firecrawl.dev/v2/crawl with url=https://www.postgresql.org/docs/17/tutorial.html, limit=25 and the documented includePaths regex path filter ["^/docs/17/tutorial"]; request.json is written before the POST and the cap/filter are module constants never changed (single run in transcript, no retuning). crawl_start_response.json shows HTTP 200 with job id 01a1e64f-8d8d-7121-8278-634f55919d5a; poll_log.json shows async polling scraping->completed (24/24). crawl_full_response.json holds the real response (status completed, total 24, creditsUsed 24, createdAt/completedAt timestamps, 24 page objects with 4.8k-12k chars of genuine PostgreSQL markdown and statusCode 200). My own re-check of that file: 24 pages, 24 distinct sourceURLs, zero outside https://www.postgresql.org/docs/17/tutorial, within 10-25. run_output.txt prints the count plus each URL and title, matching the saved JSON exactly, and the titles match each page's own h1 (title_verification.txt) and known PostgreSQL 17 tutorial structure. No local filtering, per-page fetching or link-following; no mocked data.

Attempt record

  1. 0:54Answered from saved key: Firecrawl API keyno wait
  2. —First success verified

Attempt artifacts saved.