Benchmark report

Task verified

12m 57s elapsed

The saved attempt met the documented success criterion.

Experience score

100/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs S, Current SSInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
The PEP 668 externally-managed-environment pip failure forcing a venv, plus a self-inflicted run_id key-path error when scripting the follow-up curl check.

Where the time went

12:58 agent time
  • Starting0:34
  • Research2:44
  • Attempt7:54
  • Verify1:46
12m 57sTotal elapsed
1:23Time to first success
0:00Waited on the owner

Judge reason

Saved code evidence/extract_pg_isolation.py sends the fixed URL https://www.postgresql.org/docs/17/transaction-iso.html plus a task_spec.output_schema (type json) with the required levels array fields to Parallel's documented Task API via the official parallel-web 1.3.7 SDK (client.task_run.create -> POST /v1/tasks/runs, x-api-key), with source_policy restricted to postgresql.org. Transcript line 36/39 shows the live run executing: run_id trun_15c2dc02ef04410fa1652f9a532910a7, status completed, and the extracted object printed to stdout (run_base_stdout.txt). evidence/response.json is the product's full response including Parallel's own basis/citations quoting Table 13.1 rows from the PostgreSQL page, proving the product performed the extraction (no local scraping or own-model call). An independent GET /v1/tasks/runs/{run_id} returned HTTP 200 (http_status.txt, http_headers_get_run.txt with cf-ray/x-request-id, get_run_body.json status completed). The returned object has exactly four levels with all 16 cells matching Table 13.1 (Read uncommitted: Allowed, but not in PG/Possible/Possible/Possible; Read committed: Not possible/Possible/Possible/Possible; Repeatable read: Not possible/Not possible/Allowed, but not in PG/Possible; Serializable: all Not possible), confirmed by verify.py output.

Attempt record

  1. 6:32Answered from saved key: Parallel API key (PARALLEL_API_KEY)no wait
  2. —First success verified

Attempt artifacts saved.