Benchmark report

Task verified

7m 33s elapsed

The saved attempt met the documented success criterion.

Experience score

91/ 100

Agent-Ready
  • BInstall
  • SAuth
  • SExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install B, Auth S, Execute S, Docs A, Current SBInstallSAuthSExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Waiting out 12 polls (~2 min) for a 'delivered' or 'bounced' label on the message that AgentMail only ever emits per-message via webhook.

Where the time went

7:34 agent time
  • Research0:37
  • Attempt5:34
  • Verify1:22
7m 33sTotal elapsed
4:11Time to first success
0:00Waited on the owner

Judge reason

Saved run.py makes live REST calls to https://api.agentmail.to/v0 with bearer auth; evidence/api_transcript.json records 18 real HTTP 200 calls at 2026-10-29T06:14-06:16Z. Exactly one POST /v0/inboxes/evt-test-1793254442@agentmail.to/messages/send returned message_id <010001a1864b4a7c-5257c03a-f4f2-4d9b-8980-939a3996c927-000000@email.amazonses.com> (thread 3cad6e0d-...). Pull-style reads by that identifier followed: GET /inboxes/{id}/messages/{message_id} (12 polls showing labels progress ['sent'] at 06:14:03.911Z -> ['sent','received','unread'], timestamp 2026-10-29T06:14:03.581Z, updated_at 06:14:03.970Z) and GET /inboxes/{id}/events?limit=100 filtered to that message_id, returning three label.added events (sent/received/unread, event_at 06:14:03Z and 06:14:04Z, event_ids 01M634PHZR951GVMVYKC3P0000 etc.). GET /metrics/events over the send window returned message.sent 1, message.delivered 1, message.bounced []. No webhook, Svix polling, WebSocket or dashboard was used; recipient was an inbox created for the run via POST /v0/inboxes, so no real mailbox received mail. The reported status and timestamps match evidence/summary.json and the raw transcript byte-for-byte, and nothing is mocked or hard-coded. The report's stated gap (message.delivered/message.bounced are per-message only via webhook; GET /inboxes/{id}/events is documented as 'List label change events' limited to label.added/label.removed; /metrics/events has no message_id dimension) is corroborated by the saved docs pages.

Attempt record

  1. 1:43Answered from saved key: AgentMail API keyno wait
  2. —First success verified

Attempt artifacts saved.