Benchmark report

Task verified

15m 39s elapsed

The saved attempt met the documented success criterion.

Experience score

98/ 100

Agent-Ready
  • AInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth S, Execute S, Docs S, Current SAInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
The PEP 668 pip failure forcing a venv, plus a brief SDK introspection step to confirm task_group method signatures before writing the script.

Where the time went

15:40 agent time
  • Starting0:01
  • Research2:54
  • Attempt8:36
  • Verify4:08
15m 39sTotal elapsed
3:26Time to first success
0:00Waited on the owner

Judge reason

Verified against saved artifacts, not prose. code/batch_scrape.py uses the GA Task Group API (parallel-web 1.3.7): one POST /v1/tasks/groups -> job id tgrp_34a503e07e954b78a9f21f938b2d55e4 (evidence/01), then ONE add_runs call carrying all five URLs -> 5 run ids (evidence/02), then an in-code 10s poll loop on GET /v1/tasks/groups/{id} with five recorded polls going running:5 -> completed:5, is_active=false (evidence/03_poll_log.json, 04). No single-page looping and no synchronous /v1/extract call; result.json cites the Extract sync-only gap explicitly. evidence/05_group_runs_snapshot.json holds exactly 5 completed runs, one per required PostgreSQL URL, with real page markdown (e.g. 'Transactions are a fundamental concept of all database systems...') and per-run basis citations pointing at the same URLs; evidence/06 holds the canonical per-run results. I recomputed len(page_content) from 05 and got 1645/26093/23451/6344/7919, matching the printed console_output.txt and 07_summary.json exactly. Docs (docs/group-api.md) carry no beta/preview/early-access marking for Task Groups, so the stop clause did not apply. Nothing mocked or hard-coded.

Attempt record

  1. 6:46Answered from saved key: Parallel API key (PARALLEL_API_KEY)no wait
  2. —First success verified

Attempt artifacts saved.