Benchmark report

Blocked by a prerequisite

5m 7s elapsed

An external prerequisite prevented the documented task from completing. Benchmark again

Experience score

35/ 100

Critical Gating
  • AInstall
  • CAuth
  • DExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth C, Execute D, Docs S, Current SAInstallCAuthDExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
The missing Postmark account API token: both benchmark credential requests were skipped and the documented test token is rejected for /domains, so no call past 401 was possible.

Where the time went

5:07 agent time
  • Starting0:01
  • Research0:18
  • Attempt3:32
  • Verify1:17
5m 7sTotal elapsed
—Time to first success
0:00Waited on the owner

Judge reason

The task was not completed: no tpc-bench domain was ever created. Evidence (postmark-domain-task/evidence/blocker-unauthenticated-calls.txt) shows a live POST https://api.postmarkapp.com/domains for tpc-bench-e0cf6c4e.example.com returning HTTP/2 401 {"ErrorCode":10,"Message":"Request does not contain a valid Account token."}, and script-run-api-calls.json records only that single failed create step. No DNS records, verification call, status read-back or deletion response exist. Postmark's own docs (auth.md and the Domains API reference, both fetched HTTP 200 in the transcript) state every /domains endpoint requires the account-level X-Postmark-Account-Token; the documented sending-only test token is rejected for it (403 "The Postmark Test API Token may only be used on the /email endpoint.", evidence/no-test-token-alternative.txt). The agent requested the credential twice through /usr/local/bin/benchmark-request (transcript entries 10/11 and 31/32), both returning {"outcome":"skipped"}, and no token was present in the environment. Completed setup is real and unmocked: a runnable verify_domain.py implementing create -> GET records -> verifyDkim/verifyReturnPath -> GET status -> DELETE -> confirm via urllib against the live host, executed end-to-end up to the real 401. The agent correctly declined to claim success and did not call benchmark-success.

Attempt record

  1. 0:58Requested Postmark API keydeclined automatically: campaigns don't wait for people
  2. 3:50Blocked by a prerequisiteNo Postmark account API token (X-Postmark-Account-Token) was available: two benchmark-request credential requests returned outcome "skipped", and the documented test token is rejected for account-level /domains endpoints.

Attempt artifacts saved.