Benchmark report

Task verified

25m 30s elapsed

The saved attempt met the documented success criterion.

Experience score

89/ 100

Agent-Ready
  • SInstall
  • SAuth
  • AExecute
  • SDocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute A, Docs S, Current ASInstallSAuthAExecuteSDocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
Every documented container-build path was blocked or broken, forcing a hand-written static Zig server on a FROM scratch image before any deploy could run.

Where the time went

25:30 agent time
  • Research0:41
  • Attempt22:38
  • Verify2:11
25m 30sTotal elapsed
18:38Time to first success
0:00Waited on the owner

Judge reason

Saved flyctl outputs prove the full chain on tpcbench-flyrb-20261109122650-aa1cf9.fly.dev: deploy v1 (image deployment-01M7051MF3ZY0QCE1QGZGWJQ05), poll body {"version":"v1","servedAt":"2026-11-09T12:39:40.574Z"}; rebuild only the version constant and deploy v2 (deployment-01M70556DYVEQTWP9F1324A3DJ), poll {"version":"v2",...12:41:42.588Z}; rollback via the documented redeploy-previous-artifact path `fly deploy -a <app> --image <v1 image>` run from a dir with no Dockerfile/source (08-rollback-deploy.txt shows 'Searching for image ... remotely / image found', no build step), then poll {"version":"v1",...12:43:26.187Z} plus a live re-fetch at 12:43:37.190Z (differing servedAt confirms per-request computation), and `fly status` showing the app image back at the v1 deployment tag. Version is a compile-time constant in server-v1/v2.zig (diff is only APP_VERSION), not an env var. Cleanup: `fly apps destroy` for the app and for the flyctl-auto-created remote builder; apps list and Machines API return empty, hostname has no A/AAAA record, final curl http_code=000 (curl 28 timeout), flyctl reports 'Could not find App'. Token was read from a private file into FLY_API_TOKEN only; grep over all saved artifacts finds no FlyV1/fm2_/bearer material. Only non-tpcbench- resource was fly-builder-dusky-songbird-6209, auto-provisioned by `fly deploy --remote-only` and destroyed in cleanup.

Attempt record

  1. 1:18Answered from saved key: Fly.io API token (FLY_API_TOKEN)no wait
  2. —First success verified

Attempt artifacts saved.