Benchmark report

Task verified

4m 11s elapsed

The saved attempt met the documented success criterion.

Experience score

100/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs S, Current SSInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
The scrape response returned an empty metadata object (no title), so the agent re-ran with a custom heuristic to derive the paper title from the markdown text.

Where the time went

4:11 agent time
  • Research0:17
  • Attempt2:27
  • Verify1:27
4m 11sTotal elapsed
1:00Time to first success
0:00Waited on the owner

Judge reason

Saved code /workspace/home/pdf-parse/parse_pdf.py POSTs {"url":"https://arxiv.org/pdf/1706.03762v7","formats":{"markdown":true},"maxAgeMs":0} to the documented endpoint https://api.context.dev/v1/web/scrape with a Bearer key read from a private file (stdlib urllib only; grep of imports shows json/os/sys/urllib, no local PDF library). response_status_headers.json shows HTTP 200 with live vendor headers (Cf-Ray a5b737d53bf5e8e9-PDX, X-Request-Id b545ad12-6f1b-40c8-81b2-a89f16f2be00, X-Context-Cache MISS, X-Credits-Used 1) matching response_full.json's request_id and cache_metadata {status: miss}. markdown.data is 39,866 chars (>20,000) of genuine paper text (abstract, Table complexity rows, Figure 5 caption) containing 'Attention Is All You Need'; parsed_markdown.md is byte-identical. Program stdout prints 'Paper title: Attention Is All You Need' and 'Character length of returned text: 39866', matching the response exactly. Docs fetched in-transcript carry no beta/preview/enablement marking for this endpoint.

Attempt record

  1. 0:52Answered from saved key: Context.dev API keyno wait
  2. —First success verified

Attempt artifacts saved.