Benchmark report

Task verified

3m 0s elapsed

The saved attempt met the documented success criterion.

Experience score

96/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current SSInstallSAuthSExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Essentially nothing: the whole run took 37s; the only snag was the docs' conflicting API host (agent.tinyfish.ai vs api.search.tinyfish.ai).

Where the time went

3:01 agent time
  • Starting0:01
  • Research0:32
  • Attempt1:22
  • Verify1:07
3m 0sTotal elapsed
0:38Time to first success
0:00Waited on the owner

Judge reason

Saved code /workspace/home/tinyfish-search/search.py sends the exact query "PostgreSQL transaction isolation levels" via GET https://api.search.tinyfish.ai with the documented X-API-Key header (key from benchmark-request, never printed). Live evidence: search_response_headers.json shows http_status 200, Server uvicorn, X-Request-Id 1f441f11-df34-4823-b398-85a89412865e, X-Ratelimit-Remaining 999, Date Mon, 16 Nov 2026 15:28:09 GMT, with X-Middleware-Rewrite /v1/search?query=PostgreSQL+transaction+isolation+levels&location=US. search_response.json has the documented shape (query, results, total_results, page) with 10 results carrying position/site_name/title/snippet/url. run_output.txt prints first result title "PostgreSQL: Documentation: 8.0: Transaction Isolation", absolute URL http://postgresql.org/docs/8.0/interactive/transaction-iso.html, and a 196-char nonempty snippet; I cross-checked all three against the saved JSON and they match. No mocking, scraping or hard-coded data; transcript step 18 shows the single real HTTP run producing that stdout.

Attempt record

  1. 0:58Answered from saved key: TINYFISH_API_KEYno wait
  2. —First success verified

Attempt artifacts saved.