Benchmark report

Task verified

5m 39s elapsed

The saved attempt met the documented success criterion.

Experience score

100/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs S, Current SSInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Postmark's split tokens: the server token is rejected by GET /senders, forcing a second account-token request, plus a ~12s indexing lag needing 3 polls.

Where the time went

5:39 agent time
  • Research0:20
  • Attempt3:47
  • Verify1:31
5m 39sTotal elapsed
2:18Time to first success
0:00Waited on the owner

Judge reason

Live run verified. Transcript entry 67 shows send_and_read.py executing against api.postmarkapp.com: POST /email -> HTTP 200, MessageID 288018b6-c7a0-4465-8439-81b97abd3439, To test@blackhole.postmarkapp.com (Postmark's documented test recipient, shown in evidence/02-send-request.json). The pull read GET /messages/outbound/{id}/details polled 3 times (422 not-yet-indexed, then 200 Status='Sent' with no events, then 200 with MessageEvents [{Type: Delivered, ReceivedAt: 2026-11-05T16:35:27.0000000-05:00, Details.DeliveryMessage: 'smtp;250 2.0.0 OK (sandbox blackhole)'}]); the saved evidence/05-details-final.json and 04-details-polls.json match that stdout byte-for-byte, including the full MIME Body with X-PM-Message-Id matching the send. A derived-filter read (GET /messages/outbound?recipient=...&subject=...) returned TotalCount 1 with the same MessageID and Status 'Sent', and GET /bounces?emailFilter=... returned 200 {TotalCount:0}. Sender rick@promptingco.com was discovered via GET /senders (Confirmed true), not guessed. Code in pm.py/send_and_read.py writes evidence straight from real urllib responses - nothing mocked or hard-coded; no inbox was created so no deletion applies. REPORT.md's status/timestamp fields match the saved responses exactly.

Attempt record

  1. 1:52Answered from saved key: Postmark Account API Tokenno wait
  2. —First success verified

Attempt artifacts saved.