Benchmark report

Task verified

4m 29s elapsed

The saved attempt met the documented success criterion.

Experience score

94/ 100

Agent-Ready
  • AInstall
  • SAuth
  • SExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth S, Execute S, Docs A, Current SAInstallSAuthSExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Only trivial drag: a guessed auth docs URL 404'd and the script's three 5-second status polls added ~15s; the whole run finished in 91 seconds.

Where the time went

4:30 agent time
  • Research0:29
  • Attempt2:28
  • Verify1:32
4m 29sTotal elapsed
1:31Time to first success
0:00Waited on the owner

Judge reason

Saved code (agentmail-domain/run_domain_lifecycle.py) makes real urllib HTTP calls to https://api.agentmail.to/v0 with Bearer auth read from the benchmark-provided key file; nothing is mocked or hard-coded. evidence/run_output.txt (byte-identical to the transcript tool result at the run step) and evidence/api_transcript.json show 8 live calls: POST /v0/domains -> 200 returning domain_id=tpc-bench-0aa157f0.example.com (8 lowercase hex, generated by secrets.token_hex(4)) with status NOT_STARTED and 5 DNS records; GET /v0/domains/{id} -> 200 with the same 5 records each status=MISSING; POST /v0/domains/{id}/verify -> 204; three polled GETs -> 200 with status=PENDING, reason=dns_records_missing; DELETE /v0/domains/{id} -> 204 empty body; post-delete GET -> 404 NotFoundError 'Domain not found'. REPORT.md lists all 5 records with type, name, value and priority (DKIM TXT, 2 MX, SPF TXT, DMARC TXT) and reports the final status as PENDING, explicitly stating the API never returned VERIFIED. No dashboard or SMTP was used; server-side timestamps (08:15:10-08:15:31Z) and org/pod UUIDs corroborate live responses.

Attempt record

  1. 1:06Answered from saved key: AGENTMAIL_API_KEYno wait
  2. —First success verified

Attempt artifacts saved.