The Prompting Company

Benchmark report

Task verified

8m 28s elapsed

The saved attempt met the documented success criterion.

Experience score

79/ 100

Needs Alignment
  • SInstall
  • AAuth
  • AExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth A, Execute A, Docs A, Current SSInstallAAuthAExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Asynchronous deletion: DELETE returned 202 with no body and the inbox briefly returned undocumented 409 resource_deleting, forcing polling logic and two extra full re-runs.

Where the time went

8:28 agent time
  • Research0:24
  • Attempt7:01
  • Verify1:02
8m 28sTotal elapsed
6:10Time to first success
0:00Waited on the owner

Judge reason

Saved code agentmail-task/agentmail_inbox_lifecycle.py uses the official agentmail Python SDK 2.0.22 (venv install confirmed in transcript) against the documented https://api.agentmail.to/v0 endpoints with a real Bearer key obtained via benchmark-request (outcome: provided). Runtime evidence (evidence/run_output.txt and evidence/api_responses.json, matching transcript idx 108) shows POST /v0/inboxes -> HTTP 200 with raw body printing inbox_id and deliverable address kindmanager377@agentmail.to, then DELETE /v0/inboxes/kindmanager377@agentmail.to -> HTTP 202 (documented empty success body), confirmed by GET of that inbox -> HTTP 404 not_found and GET /v0/inboxes -> 200 listing only two unrelated inboxes with the deleted id absent. Nothing is mocked or hard-coded: raw httpx response text, organization/pod ids, and timestamps are captured at runtime, and earlier failing runs (409 resource_deleting, a serialization traceback) are preserved in the transcript, showing genuine live API behavior.

Attempt record

  1. 0:50Answered from saved key: AgentMail API keyno wait
  2. —First success verified

Attempt artifacts saved.