Benchmark report
Task verified
3m 54s elapsed
The saved attempt met the documented success criterion.
96/ 100
Agent-Ready- SInstall
- SAuth
- SExecute
- ADocs
- SCurrent
Missing advanced_settings schema: the API-reference page and openapi.json 404'd, so the agent read parallel-web source to learn full_content takes a bool.
Where the time went
3:54 agent time- Research0:16
- Attempt2:10
- Verify1:28
Judge reason
Live call verified: code/extract_pdf.py posts urls=["https://arxiv.org/pdf/1706.03762v7"] with advanced_settings {full_content: true} to Parallel's documented POST https://api.parallel.ai/v1/extract via parallel-web 1.3.7 (x-api-key from a benchmark-provided key file). Transcript idx 45-46 shows the run and its stdout; evidence/extract_response.json holds a genuine response (extract_id extract_6000a3ad765f53d664fc253794437285, session_id, usage sku_extract_excerpts, acquired_at 2026-11-11, errors []). I independently re-parsed it: results[0].url equals the requested PDF URL, title is "Attention Is All You Need", full_content is 41522 chars (>20000) and contains the title; the text is authentic paper body (e.g. the d_model=512 / N=6 decoder passage). full_content.md sha256 5914ec9b... matches both the file and the response field, and printed title/length match exactly. No local PDF download or local parsing appears anywhere in the transcript or file tree. Docs fetch confirms /v1/extract is GA (only /v1beta/extract is legacy), so no beta/preview marking applies. Minor nit: commands.txt lists the curl script under commands run though the transcript shows only chmod, not execution; the SDK run alone satisfies all criteria.
Attempt record
- 0:47Answered from saved key: Parallel API keyno wait
- —First success verified
Attempt artifacts saved.