Benchmark report

Task verified

13m 6s elapsed

The saved attempt met the documented success criterion.

Experience score

90/ 100

Agent-Ready
  • SInstall
  • SAuth
  • AExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute A, Docs S, Current SSInstallSAuthAExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Context.dev queued the batch for roughly 8.5 minutes (100 polls) before starting, though actual scraping of all five URLs took under a second.

Where the time went

13:07 agent time
  • Research0:24
  • Attempt10:59
  • Verify1:44
13m 6sTotal elapsed
10:11Time to first success
0:00Waited on the owner

Judge reason

Saved code /workspace/home/context-batch/batch_scrape.py submits all five PostgreSQL URLs in ONE POST https://api.context.dev/v1/batch/submit (01-submit-request.json shows the 5-URL array, itemIds pg-1..pg-5) and the live 200 response (02-submit-response.json) returns one job id batch_6b99c421a36f429c91110a970ed91b83 with status=pending plus server-side request_id and credits_remaining. The code then polls GET /v1/batch/{id} in a loop until a terminal status: 03-poll-history.json holds 102 timestamped 200 polls on that same id (pending -> queued x100 -> completed, 13:34:35Z to 13:43:16Z). 04-final-batch.json shows status=completed, progress {succeeded:5, failed:0, pending:0}, credits net 5, signed GCS output file with items:5. GET /v1/batch/{id}/results returned one page (has_more=false) with 5 records, each status=ok with real PostgreSQL 17 markdown. I recomputed the lengths from the raw saved bodies (07-markdown-bodies.json): 4917/29673/22778/10384/11340 chars, exactly matching the printed run-output.txt, 06-summary.json and result.json. No looped single-page /web/scrape calls and no synchronous multi-URL call appear anywhere. Nothing is mocked or hard-coded.

Attempt record

  1. 1:06Answered from saved key: Context.dev API keyno wait
  2. —First success verified

Attempt artifacts saved.