Benchmark report

Task verified

5m 12s elapsed

The saved attempt met the documented success criterion.

Experience score

96/ 100

Agent-Ready
  • AInstall
  • SAuth
  • SExecute
  • SDocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth S, Execute S, Docs S, Current AAInstallSAuthSExecuteSDocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
pip's PEP 668 block forcing a venv, plus a ~10s wait and second poll before the sent event appeared on the events endpoint.

Where the time went

5:12 agent time
  • Starting0:01
  • Research0:24
  • Attempt3:15
  • Verify1:33
5m 12sTotal elapsed
1:59Time to first success
0:00Waited on the owner

Judge reason

Saved code and raw responses prove the full task against the live AgentMail API. send_and_read_events.py (urllib over https://api.agentmail.to/v0, bearer key from benchmark-request) created a recipient inbox for the run (POST /v0/inboxes -> 200, evt-recipient-1793915463@agentmail.to) and a sender inbox on the default domain after GET /v0/domains returned {count:0} (evidence/00_discover_sender.json). It sent exactly one message: POST /v0/inboxes/evt-sender-1793915463@agentmail.to/messages/send -> HTTP 200, message_id <010001a1adb1af95-968ae94c-f304-4e25-887a-7b10547bae61-000000@email.amazonses.com>, with the recipient shown in the saved request body. It then read the pull endpoint GET /v0/inboxes/{inbox_id}/events?limit=100 -> 200 and filtered on that message_id, returning event_id 01M6PV3B98CTB9SZ814F060000, event_type label.added, label 'sent', event_at 2026-11-05T21:51:05.000Z, created_at 2026-11-05T21:51:06.276Z; the recipient inbox's events endpoint returned 'received' and 'unread' for the same message_id, and GET /v0/inboxes/{id}/messages/{message_id} -> 200 with labels ['sent'] and timestamp 2026-11-05T21:51:05.507Z. No webhook or dashboard was used. The report's table matches evidence/01_send_and_read_events.json and 02_cleanup_delete_inboxes.json byte for byte, and the console log shows two real polls (matching=0 then 1), so nothing is mocked or hard-coded. Cleanup is included: DELETE /v0/inboxes/{id} -> 202 for both inboxes, verified settling to HTTP 404 not_found in 03_verify_inboxes_deleted.json. The report honestly notes only label.added/label.removed event types exist on this pull endpoint and that no bounce occurred, inventing no delivery data.

Attempt record

  1. 0:56Answered from saved key: AgentMail API keyno wait
  2. —First success verified

Attempt artifacts saved.