The Prompting Company

Benchmark report

Task verified

9m 52s elapsed

The saved attempt met the documented success criterion.

Experience score

96/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current SSInstallSAuthSExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
No local Docker daemon, so the agent packaged server.py as a base64 env var run via fly.toml [processes] on the stock python:3.12-alpine image instead of a normal build.

Where the time went

9:52 agent time
  • Research0:16
  • Attempt7:46
  • Verify1:50
9m 52sTotal elapsed
3:39Time to first success
0:00Waited on the owner

Judge reason

Saved evidence proves the full flow on Fly.io's documented CLI. evidence/01-apps-create.txt shows `fly apps create tpcbench-20261106223125-3af6d85e --org personal` (org resolved via `fly orgs list --json`); 02-deploy.txt shows `fly deploy --config code/fly.toml --image python:3.12-alpine` launching machine 28744227f36128 and the platform-assigned URL https://tpcbench-20261106223125-3af6d85e.fly.dev/ on fly.dev. 03-poll.txt ends HTTP 200. 04-two-fetches.txt has two HTTP/2 200 responses from Fly (server: Fly/366e4bf88, distinct fly-request-ids) carrying the same marker tpcbench-marker-d2dc4f9e1a0098c8d382c76c7f105674 with differing servedAt (22:33:21.779949Z vs 22:33:24.044149Z), matching code/server.py which computes datetime.now(timezone.utc) per request — no mock or hard-coded body. 05-destroy.txt shows `fly apps destroy ... --yes` -> Destroyed and `fly apps list --json` -> []; 06/07 show the URL timing out (HTTP 000, 4 attempts over ~90s), `fly status` -> Could not find App, and Machines API GET /v1/apps/<app> -> 404 app not found. Only the tpcbench-prefixed app was created (apps list was [] before and after). Grep of home and transcript finds no FlyV1/fm2_ token value; the token was kept in /tmp/flypriv outside home and deleted.

Attempt record

  1. 0:51Answered from saved key: FLY_API_TOKENno wait
  2. —First success verified

Attempt artifacts saved.