Benchmark report

Task verified

9m 16s elapsed

The saved attempt met the documented success criterion.

Experience score

100/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs S, Current SSInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Essentially nothing: docs read, key request, and the live 200 extraction completed in ~50s total with no retries or errors.

Where the time went

9:16 agent time
  • Starting0:41
  • Research0:56
  • Attempt5:06
  • Verify2:34
9m 16sTotal elapsed
0:53Time to first success
0:00Waited on the owner

Judge reason

Saved code /workspace/home/context-extract/extract.py POSTs https://api.context.dev/v1/web/scrape with Authorization: Bearer <key>, url=https://www.postgresql.org/docs/17/transaction-iso.html, formats.json=true and jsonParams.schema (levels array of objects with name/dirtyRead/nonrepeatableRead/phantomRead/serializationAnomaly) — the product's documented structured-extraction operation, with no local HTML/markdown parsing and no external LLM call. response.json records http_status 200 with a genuine product payload (json.requested true, json.success true, request_id eb571b6b-bc98-4ddf-ba86-d039237a1972, key_metadata.credits_consumed 5 / credits_remaining 4713414, page metadata and headings), and the transcript shows the script actually executed (exit=0, 'HTTP status: 200' in stderr.txt, object echoed to stdout.txt). json.data has exactly four levels matching Table 13.1: Read uncommitted = Allowed, but not in PG/Possible/Possible/Possible; Read committed = Not possible/Possible/Possible/Possible; Repeatable read = Not possible/Not possible/Allowed, but not in PG/Possible; Serializable = Not possible x4. Agent also checked docs.context.dev/scrape/json for beta/preview markings and found none. No fabrication or mocking in evidence.

Attempt record

  1. 4:07Answered from saved key: CONTEXT_DEV_API_KEYno wait
  2. —First success verified

Attempt artifacts saved.