Benchmark report

Task verified

6m 27s elapsed

The saved attempt met the documented success criterion.

Experience score

98/ 100

Agent-Ready
  • AInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth S, Execute S, Docs S, Current SAInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Up-front doc reading on task spec and processor choice, plus the ~61s the batch itself took to reach completed:5 across seven 10s polls.

Where the time went

6:27 agent time
  • Research0:23
  • Attempt4:30
  • Verify1:34
6m 27sTotal elapsed
3:29Time to first success
0:00Waited on the owner

Judge reason

Saved code parallel-batch/batch_scrape.py uses Parallel's documented async Task Group batch API: POST /v1/tasks/groups returned one job id tgrp_fcf4f1e49e1f471aaeaa360021935778 (01_create_group.json, HTTP 200), then ONE POST /v1/tasks/groups/{id}/runs carried all five PostgreSQL URLs and returned 5 run_ids (02_add_runs.json, HTTP 200, num_task_runs=5). The code polls GET /v1/tasks/groups/{id} in a loop; 03_poll_history.json and run_stdout.txt show 7 polls over 61s going running:5 -> completed:5, is_active=false (04_final_group_status.json). 05_run_results_full.json holds 5 results, one per URL, each HTTP 200 / run_status completed with nonempty real page text (5247, 29139, 22798, 10691, 11466 chars) whose citations name the matching PostgreSQL 17 pages; I independently recomputed len(content) and every value matches the printed table and 06_report.json. No single-page loop, no synchronous /v1/extract call, no mocked or hard-coded data. Task Group is listed as GA in docs llms.txt with no beta/preview marking.

Attempt record

  1. 1:27Answered from saved key: Parallel API key (PARALLEL_API_KEY)no wait
  2. —First success verified

Attempt artifacts saved.