Benchmark report

Task verified

15m 40s elapsed

The saved attempt met the documented success criterion.

Experience score

99/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • SDocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs S, Current ASInstallSAuthSExecuteSDocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
The docs' camelCase JSON body vs the Python SDK's snake_case kwargs caused a TypeError on the first run, plus extra work to exclude the 'Hello World!' string embedded in the page's inline script.

Where the time went

15:40 agent time
  • Starting3:17
  • Research1:44
  • Attempt6:29
  • Verify4:10
15m 40sTotal elapsed
1:55Time to first success
0:00Waited on the owner

Judge reason

Evidence shows a live authenticated POST to api.context.dev/v1/web/scrape via the official Python SDK context.dev 2.49.0, with the documented sharedParams.actions sequence captured from the actual wire body (evidence/request.json): perform 'Click the Start button' -> waitFor '#finish h4' -> wait 1000ms, then capture. The saved response (evidence/response.json) is HTTP 200 with request_id 82997a9e-3e61-4055-92a9-06d8a03880fc, cache_metadata miss, credits_consumed 2; its markdown contains '#### Hello World!' and metadata.headings includes 'Hello World!'. The returned HTML is the post-click rendered DOM (#start display:none, #loading display:none, <div id="finish" style=""><h4>Hello World!</h4></div>), which a static fetch cannot produce. run-output.txt prints 'Revealed heading: Hello World!' matching the response, and the script asserts against the rendered #finish element after stripping <script> blocks, so the literal string inside the page's inline script is not what was matched. A control run through the same API without actions (control-no-actions-output.txt/-response.json) returns markdown containing 'Start' and no 'Hello World!', confirming the click caused the reveal. No local browser: pip list showed no playwright/selenium/puppeteer and only api.context.dev was called. Docs (fetched browser-actions.md) mark actions as requiring a paid plan - a pricing tier, not beta/preview/early-access/on-request - so the stop condition did not apply.

Attempt record

  1. 7:02Answered from saved key: CONTEXT_DEV_API_KEYno wait
  2. —First success verified

Attempt artifacts saved.