Benchmark report

Task verified

15m 24s elapsed

The saved attempt met the documented success criterion.

Experience score

99/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • SDocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs S, Current ASInstallSAuthSExecuteSDocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
Almost nothing; only the hosted screenshot URL could not be downloaded (sandbox proxy 403), so the PNG was not saved locally.

Where the time went

15:24 agent time
  • Starting0:26
  • Research2:12
  • Attempt9:49
  • Verify2:57
15m 24sTotal elapsed
1:12Time to first success
0:00Waited on the owner

Judge reason

Saved code firecrawl-task/scrape_with_actions.py POSTs to https://api.firecrawl.dev/v2/scrape with the documented action sequence [wait #start button, click #start button, wait #finish, wait 2000ms, screenshot] using only stdlib urllib (no local browser). response_full.json is a live Firecrawl response: success true, scrapeId 01a1c747-c0aa-71ba-8c0c-73620c60ca0c, creditsUsed 1, proxyUsed basic, and a Firecrawl-hosted signed screenshot URL. Its HTML shows the post-click DOM '<div id="finish" style=""><h4>Hello World!</h4></div>' with #start toggled to display:none; markdown also contains 'Hello World!'. run_output.txt shows HTTP 200 and 'REVEALED HEADING TEXT: Hello World!', matching the response. A control run with no actions (control_response.json/control_output.txt) returned HTML with #start visible and no 'Hello World!' and no #finish, proving the click executed in Firecrawl's hosted browser. Transcript shows the run (HTTP 200) before benchmark-success was called with response_full.json.

Attempt record

  1. 8:13Answered from saved key: FIRECRAWL_API_KEYno wait
  2. —First success verified

Attempt artifacts saved.