Benchmark report

Blocked by a prerequisite

5m 17s elapsed

An external prerequisite prevented the documented task from completing. Benchmark again

Experience score

38/ 100

Critical Gating
  • SInstall
  • CAuth
  • DExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth C, Execute D, Docs S, Current SSInstallCAuthDExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
The supplied account token was invalid (401 ErrorCode 10 everywhere), compounded by three malformed benchmark-request calls that exhausted the input budget before a replacement could be asked for.

Where the time went

5:17 agent time
  • Research0:15
  • Attempt3:31
  • Verify1:31
5m 17sTotal elapsed
—Time to first success
0:00Waited on the owner

Judge reason

No part of the task's core flow succeeded. postmark_domain_verify.py implements the documented flow (POST /domains, GET /domains/{id}, PUT verifyDkim, PUT verifyReturnPath, re-read status, DELETE), but the single live run (evidence/run-output.txt and create-attempt-output.txt) shows it aborting on the first call: 'Create domain -> HTTP 401 {"ErrorCode":10,"Message":"Request does not contain a valid Account token."}' for tpc-bench-a4fd6b3a.example.com. No domain ID, no DNS records, no status read-back and no delete response exist anywhere in the artifacts; result.json honestly records domain_created=false, dns_records=[], final_status=null, deleted=false, and the agent never called benchmark-success. The credential supplied by the benchmark mechanism (transcript line 42, outcome 'provided', 46-byte non-UUID value) is rejected by the live api.postmarkapp.com under X-Postmark-Account-Token, X-Postmark-Server-Token and Authorization, and evidence/auth-diagnostics.txt shows a bogus all-zero control token producing a byte-identical 401, while the documented POSTMARK_API_TEST sandbox token returns 403 'may only be used on the /email endpoint'. Setup was completed correctly: official docs (auth.md, domains-api, .well-known/api-catalog) were located, the account-level auth header was identified from the docs, and correct runnable code was written and executed against the real API.

Attempt record

  1. 1:05Answered from saved key: Postmark API keyno wait
  2. 3:46Blocked by a prerequisiteThe Postmark account API token issued through the benchmark credential mechanism is not valid for api.postmarkapp.com - every account-level Domains API call returns 401 ErrorCode 10, identical to a deliberately bogus control token, and Postmark issues account tokens only through the dashboard UI, which the task forbids.

Attempt artifacts saved.