Benchmark report
Task verified
12m 57s elapsed
The saved attempt met the documented success criterion.
100/ 100
Agent-Ready- SInstall
- SAuth
- SExecute
- SDocs
- SCurrent
The PEP 668 externally-managed-environment pip failure forcing a venv, plus a self-inflicted run_id key-path error when scripting the follow-up curl check.
Where the time went
12:58 agent time- Starting0:34
- Research2:44
- Attempt7:54
- Verify1:46
Judge reason
Saved code evidence/extract_pg_isolation.py sends the fixed URL https://www.postgresql.org/docs/17/transaction-iso.html plus a task_spec.output_schema (type json) with the required levels array fields to Parallel's documented Task API via the official parallel-web 1.3.7 SDK (client.task_run.create -> POST /v1/tasks/runs, x-api-key), with source_policy restricted to postgresql.org. Transcript line 36/39 shows the live run executing: run_id trun_15c2dc02ef04410fa1652f9a532910a7, status completed, and the extracted object printed to stdout (run_base_stdout.txt). evidence/response.json is the product's full response including Parallel's own basis/citations quoting Table 13.1 rows from the PostgreSQL page, proving the product performed the extraction (no local scraping or own-model call). An independent GET /v1/tasks/runs/{run_id} returned HTTP 200 (http_status.txt, http_headers_get_run.txt with cf-ray/x-request-id, get_run_body.json status completed). The returned object has exactly four levels with all 16 cells matching Table 13.1 (Read uncommitted: Allowed, but not in PG/Possible/Possible/Possible; Read committed: Not possible/Possible/Possible/Possible; Repeatable read: Not possible/Not possible/Allowed, but not in PG/Possible; Serializable: all Not possible), confirmed by verify.py output.
Attempt record
- 6:32Answered from saved key: Parallel API key (PARALLEL_API_KEY)no wait
- —First success verified
Attempt artifacts saved.