Benchmark report

Task verified

13m 10s elapsed

The saved attempt met the documented success criterion.

Experience score

100/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs S, Current SSInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Essentially nothing: docs-to-verified-success in ~53s on the first call; the only nuance was needing advanced_settings.full_content for full text vs excerpts.

Where the time went

13:11 agent time
  • Starting0:03
  • Research1:50
  • Attempt5:14
  • Verify6:04
13m 10sTotal elapsed
0:53Time to first success
0:00Waited on the owner

Judge reason

Transcript shows the agent read the official Extract docs (quickstart + POST /v1/extract reference, confirming no beta/preview label on Extract), requested one API key via benchmark-request, then ran extract_pdf.py posting {"urls":["https://arxiv.org/pdf/1706.03762v7"], advanced_settings.full_content} to https://api.parallel.ai/v1/extract with an x-api-key header. Live output: HTTP 200, extract_id extract_370eddd81ce58a2c49f8d19a11eb05c6, TITLE 'Attention Is All You Need', LENGTH 41522. Independent re-parse of the saved response.json confirms results[0].url == the arXiv PDF URL, title == 'Attention Is All You Need', full_content length 41522 (>20,000) containing that title, errors [], usage sku_extract_excerpts; full_content.md byte-matches the response and reads as genuine paper text (Vaswani et al. authors through Figure 5, page 15). No local PDF download or local parsing appears anywhere in the transcript; parsing was server-side by the product. Printed title/length match the saved response exactly.

Attempt record

  1. 5:15Answered from saved key: PARALLEL_API_KEYno wait
  2. —First success verified

Attempt artifacts saved.