Benchmark report

Task verified

14m 21s elapsed

The saved attempt met the documented success criterion.

Experience score

99/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • SDocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs S, Current ASInstallSAuthSExecuteSDocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
Polling 60s (4 polls) for the triggered run's completion event, plus one PEP 668 pip failure forcing a venv before the SDK installed.

Where the time went

14:22 agent time
  • Starting1:38
  • Research1:45
  • Attempt7:05
  • Verify3:54
14m 21sTotal elapsed
2:56Time to first success
0:00Waited on the owner

Judge reason

Saved runnable code (parallel-monitor/monitor_lifecycle.py) uses the real parallel-web 1.3.7 SDK against https://api.parallel.ai with x-api-key, and evidence/run.log + evidence/transcript.json show the live lifecycle: POST /v1/monitors returned monitor_9d24275f626c4c5a90c455400f819a91 (status active, printed as 'MONITOR ID: ...'); GET /v1/monitors/{id} read back frequency=1h (docs fetched in transcript state frequency must be between 1h and 30d, so 1h is the shortest allowed) with settings.query containing the full target URL https://www.postgresql.org/docs/17/release-17.html and source_policy.include_domains=['postgresql.org/docs/17/release-17.html'] (server canonicalized away the 'www.' the code sent, which is why the code's own exact-string check logged false); POST /v1/monitors/{id}/trigger returned 204 and last_run_at advanced from 20:55:00.610986Z to 20:55:00.837480Z; GET /v1/monitors/{id}/events?include_completions=true returned after 4 polls one run result {event_type: completion, timestamp 2026-11-10T20:55:00.871374Z}. Teardown ran in a finally block: POST /v1/monitors/{id}/cancel returned status=cancelled and an independent GET re-confirmed status=cancelled; the agent cited the fetched cancel-monitor doc stating no DELETE endpoint is documented and cancellation is irreversible. Docs fetches recorded no beta/preview/early-access marking on the Monitor API, so the stop-and-report clause does not apply. No mocked or hard-coded responses; server-generated ids and timestamps throughout.

Attempt record

  1. 5:48Answered from saved key: PARALLEL_API_KEYno wait
  2. —First success verified

Attempt artifacts saved.