Benchmark report

Task verified

3m 50s elapsed

The saved attempt met the documented success criterion.

Experience score

96/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current SSInstallSAuthSExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Server API token cannot list sender signatures or domains (404/401), so the agent had to infer a permitted From address from GET /messages/outbound history.

Where the time went

3:50 agent time
  • Research0:20
  • Attempt2:19
  • Verify1:11
3m 50sTotal elapsed
1:21Time to first success
0:00Waited on the owner

Judge reason

Saved code postmark-task/send_email.py POSTs to https://api.postmarkapp.com/email with the documented X-Postmark-Server-Token header (token read from the benchmark-request file, never hard-coded). Recipient is Postmark's documented blackhole address test@blackhole.postmarkapp.com; sender rick@promptingco.com was discovered live via GET /messages/outbound (transcript idx 33-34, evidence/discovery2.json), after /sender-signatures returned 404 and /domains 401 for a Server token. Transcript idx 43-44 shows the live run output: 'POST /email -> HTTP 200' with body {"ErrorCode":0,"Message":"OK","MessageID":"d8c85482-186a-47b4-ad66-33c6c06301f9","SubmittedAt":"2026-11-02T07:37:10.4823144Z","To":"test@blackhole.postmarkapp.com"}, matching evidence/send_email_stdout.txt and evidence/send_email_response.json (which records real Postmark response headers: X-Postmark-Server 19755949, Ratelimit-Remaining 4999). The saved request body (Subject 'Order #PC-10427 confirmed', TextBody, MessageStream outbound) is byte-identical to the payload literal in send_email.py. Independent confirmation: GET /messages/outbound/d8c85482-.../details returned HTTP 200 with Status 'Sent', Delivered event 'smtp;250 2.0.0 OK (sandbox blackhole)', and Subject/TextBody matching the code (evidence/verify_message.json). HTTP API used, not SMTP or dashboard; exactly one send. No mocking or fabrication.

Attempt record

  1. 0:44Answered from saved key: Postmark Server API tokenno wait
  2. —First success verified

Attempt artifacts saved.