Benchmark report

Task verified

12m 36s elapsed

The saved attempt met the documented success criterion.

Experience score

93/ 100

Agent-Ready
  • AInstall
  • SAuth
  • SExecute
  • ADocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth S, Execute S, Docs A, Current AAInstallSAuthSExecuteADocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
A 404 on the guessed docs path /batches/status-and-results plus missing `requests`, both resolved in one step each; total run ~106s.

Where the time went

12:36 agent time
  • Starting1:50
  • Research2:00
  • Attempt5:47
  • Verify2:59
12m 36sTotal elapsed
1:46Time to first success
0:00Waited on the owner

Judge reason

Saved code context-batch/batch_scrape.py makes ONE POST https://api.context.dev/v1/batch/submit with input.mode=scrape and all five PostgreSQL URL records, then polls GET /v1/batch/{id} in a loop until a terminal status, then pages GET /v1/batch/{id}/results. Evidence confirms it ran live: 01_submit_response.json shows HTTP 200, id batch_27c13c14c79299aa53135b8578a49b12, input.submitted=5, duplicates=0, invalid=0, status pending, credits_remaining 4714253; 02_poll_history.json holds two real 200 polls (pending -> completed) with distinct request_ids; 03_final_status.json shows status completed, progress succeeded=5 failed=0 pending=0, credits net 5 and a signed GCS results file (5 items). 04_results.json holds exactly 5 records, one per requested URL, each status=ok with nonempty genuine PostgreSQL 17 markdown (mvcc-intro body contains '13.1. Introduction' and real version-switcher links). Printed values in run_stdout.txt (6212/30939/24135/11656/12617) match 05_summary.json and recomputed len() over the saved markdown exactly. No single-page loop and no synchronous multi-URL call were used; nothing was mocked or hard-coded.

Attempt record

  1. 6:11Answered from saved key: CONTEXT_DEV_API_KEYno wait
  2. —First success verified

Attempt artifacts saved.