Benchmark report

Task verified

5m 0s elapsed

The saved attempt met the documented success criterion.

Experience score

98/ 100

Agent-Ready
  • AInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth S, Execute S, Docs S, Current SAInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Waiting on the asynchronous job: 11 polls sat at completed=23/total=24 for ~55s before status flipped to completed.

Where the time went

5:00 agent time
  • Research0:14
  • Attempt3:03
  • Verify1:43
5m 0sTotal elapsed
2:11Time to first success
0:00Waited on the owner

Judge reason

Saved code firecrawl-crawl-project/crawl_pg_tutorial.py issues one server-side job: POST https://api.firecrawl.dev/v2/crawl with url=https://www.postgresql.org/docs/17/tutorial.html, limit=25, includePaths=["^/docs/17/tutorial"] (Firecrawl's documented include option, pathname regex per crawl-feature.md:552), then polls GET /v2/crawl/{id}. crawl_start_response.json shows HTTP 200 id 01a1e507-a274-73df-90b7-4df79fb9f167; crawl_status_response.json holds a 12-entry poll log (scraping -> completed, 15:44:05Z to 15:45:00Z, duration 55.5s, creditsUsed 24) and 24 page objects, all statusCode 200 with distinct per-page markdown and Firecrawl metadata (scrapeId, proxyUsed basic/stealth). I re-checked independently: 24 pages, 24 distinct sourceURLs, 0 outside https://www.postgresql.org/docs/17/tutorial, and all 24 URL+title pairs in run_output.txt match the saved response exactly; each title's section label (e.g. '2.6. Joins Between Tables') appears in that page's own markdown, 24/24. No local link-following or post-filtering; local fetches of postgresql.org were 403 at the egress proxy, consistent with server-side crawling. 24 pages is within 10-25.

Attempt record

  1. 0:52Answered from saved key: Firecrawl API keyno wait
  2. —First success verified

Attempt artifacts saved.