Benchmark report
Task verified
4m 36s elapsed
The saved attempt met the documented success criterion.
99/ 100
Agent-Ready- SInstall
- SAuth
- SExecute
- SDocs
- ACurrent
Almost nothing: first success in 119s; only minor friction was docs lacking a full-page screenshot option, requiring Playwright's own full_page parameter.
Where the time went
4:37 agent time- Research0:29
- Attempt2:44
- Verify1:23
Judge reason
Saved code (capture.py) and evidence.json show the documented TinyBrowser flow: POST https://api.browser.tinyfish.ai with X-API-Key and {"url":"https://www.postgresql.org/docs/17/transaction-iso.html"} -> 201, session tf-47aca29f-..., cdp_url wss://ip-54-241-44-51.tetra-data.production.tinyfish.io/..., then connect_over_cdp + page.screenshot(full_page=True) (the pattern on docs.tinyfish.ai/browser-api/examples.md), then DELETE -> 204. Rendering was remote: evidence records remote UA Chrome/147.0.0.0, remote title 'PostgreSQL: Documentation: 17: 13.2. Transaction Isolation', viewport 1251x609, scrollHeight 6672; no 'playwright install' was run, so no local browser binaries existed. Saved run-output.txt prints 'width=1236 height=6672'; transcript command output shows ls -l 876867 bytes and od of the file: 89 50 4e 47 0d 0a 1a 0a ... IHDR 00 00 04 d4 (1236) 00 00 1a 10 (6672), plus an independent re-read 'sig_ok=True'. Valid PNG signature, height (6672, equal to remote scrollHeight) > width (1236), printed dims match the file header. Nothing mocked or hard-coded. The PNG binary itself is absent from the copied home snapshot, but the hexdump, byte count and two independent dimension reads of the real file are saved command output, not prose.
Attempt record
- 1:35Answered from saved key: TINYFISH_API_KEYno wait
- —First success verified
Attempt artifacts saved.