Benchmark report

Task verified

24m 13s elapsed

The saved attempt met the documented success criterion.

Experience score

93/ 100

Agent-Ready
  • AInstall
  • SAuth
  • SExecute
  • ADocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth S, Execute S, Docs A, Current AAInstallSAuthSExecuteADocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
Waiting ~12 min for the slowest run (explicit-locking) to finish, plus discovering the SDK has no run-batch method and falling back to raw HTTP.

Where the time went

24:13 agent time
  • Starting1:37
  • Research0:26
  • Attempt18:52
  • Verify3:19
24m 13sTotal elapsed
14:36Time to first success
0:00Waited on the owner

Judge reason

Independently re-verified the saved evidence offline. batch_scrape.py makes ONE POST https://agent.tinyfish.ai/v1/automation/run-batch with a 5-element runs array (01_batch_submit_request.json lists exactly the five PostgreSQL 17 URLs); 02_batch_submit_response.json shows HTTP 200 with 5 run_ids and error:null, all five runs created_at 2026-11-10T21:26:46Z (atomic). The code then polls POST /v1/runs/batch in a loop; 04_poll_log.json records 70 ticks from {'PENDING':5} to {'COMPLETED':5}. 03_final_runs_full_response.json holds 5 COMPLETED runs whose API-echoed `goal` fields contain their own target URL, confirming URL-to-run_id mapping independently of array order; content lengths 1855/23411/16729/6234/7310 match run_stdout.txt byte-for-byte and match the per-URL content_*.txt files exactly (my own recomputation, not the agent's verifier), and the text is genuine PostgreSQL doc prose ('13.1. Introduction', '13.3. Explicit Locking', ...). The fetched run-batch reference explicitly states the endpoint is not marked beta/preview/early-access. No synchronous or single-page-loop call was used (/v1/automation/run and /run-async deliberately avoided). The only deviation: TinyFish's run-batch returns no batch id, so the printed job id is a local sha256 label over the ordered run_ids (recomputes to batch-f7e782dd50be8802) - the researched plan itself states the batch identity is the run_ids array, so this is a product gap honestly disclosed, not fabrication.

Attempt record

  1. 5:53Answered from saved key: TinyFish API keyno wait
  2. —First success verified

Attempt artifacts saved.