The Prompting Company

Benchmark report

Blocked by a prerequisite

5m 51s elapsed

An external prerequisite prevented the documented task from completing. Benchmark again

Experience score

55/ 100

Critical Gating
  • SInstall
  • BAuth
  • CExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth B, Execute C, Docs S, Current SSInstallBAuthCExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Three benchmark-request calls were rejected on JSON schema ('fields' shape) before a valid one, burning the request budget; then the test token's /email-only scope blocked every read.

Where the time went

5:51 agent time
  • Research0:14
  • Attempt4:02
  • Verify1:34
5m 51sTotal elapsed
—Time to first success
0:00Waited on the owner

Judge reason

Saved code (postmark-task/send-and-read.js, read-with-sdk.js) is real, unmocked, and ran: evidence/02-send-email.json shows POST https://api.postmarkapp.com/email -> HTTP 200 {ErrorCode:0, Message:'Test job accepted', MessageID 00e3cec7-215d-4f65-8b98-f00917581ba8, SubmittedAt 2026-10-20T22:29:34.795389Z} to test@blackhole.postmarkapp.com, so the send half and identifier capture are verified. The required read half never returned event or status data: evidence/03-message-details.json (GET /messages/outbound/<id>/details), 04-messages-outbound-filter.json (GET /messages/outbound?recipient=...) and 05-sdk-read.json (postmark@5.1.0 getOutboundMessageDetails) all returned HTTP 403 {ErrorCode:10,'The Postmark Test API Token may only be used on the /email endpoint.'}; run-console.log matches. No timestamp/state event payload exists, and the report correctly reports no data rather than fabricating it, and correctly states this is not an API gap (the endpoint is present in the official swagger/server.yml at line 1173 with X-Postmark-Server-Token required). Setup was otherwise complete: npm install postmark -> 5.1.0 clean, docs fetched from postmarkapp.com/auth.md and /.well-known/api-catalog. The agent requested a Server API token through /usr/local/bin/benchmark-request (returned 'skipped - No one answers requests during benchmark campaigns'), and an env/file scan showed no saved Postmark key, so the blocker is a genuinely unavailable credential; success criteria remain unmet. No benchmark-first-success.json exists and benchmark-success was never called.

Attempt record

  1. 0:55Requested Postmark API keydeclined automatically: campaigns don't wait for people
  2. 4:17Blocked by a prerequisiteNo Postmark Server API token was obtainable: benchmark-request returned 'skipped' (budget exhausted) and no saved key existed in the environment; Postmark's documented POSTMARK_API_TEST token is server-side restricted to /email, so the documented read endpoint GET /messages/outbound/{messageid}/details returned 403 ErrorCode 10.

Attempt artifacts saved.