Benchmark report

Task verified

4m 0s elapsed

The saved attempt met the documented success criterion.

Experience score

98/ 100

Agent-Ready
  • AInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth S, Execute S, Docs S, Current SAInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Only real friction was the PEP 668 externally-managed Python environment rejecting the first pip install; a venv resolved it in one follow-up command.

Where the time went

4:00 agent time
  • Research0:25
  • Attempt2:11
  • Verify1:24
4m 0sTotal elapsed
1:22Time to first success
0:00Waited on the owner

Judge reason

Saved code tinysearch-project/search_tinyfish.py sends the exact query "PostgreSQL transaction isolation levels" via the official SDK (tinyfish 0.11.0, client.search.query) to the documented operation; raw_http_evidence.py repeats it as GET https://api.search.tinyfish.ai/?query=PostgreSQL+transaction+isolation+levels with the X-API-Key header. raw_http_evidence.json records a live HTTP 200 with server-side headers that cannot be fabricated locally (Server: uvicorn, X-Request-Id ef6479d9-..., X-Ratelimit-Remaining 998, CloudFront X-Amz-Cf-Id, Date Mon, 16 Nov 2026 15:08:57 GMT). response.json holds the verbatim body in the documented shape (query, results[{position, site_name, title, snippet, url}], total_results, page) with 10 nonempty results. The printed first result in run_output.txt matches the saved body exactly: title "PostgreSQL: Documentation: 8.0: Transaction Isolation", absolute URL http://postgresql.org/docs/8.0/interactive/transaction-iso.html, nonempty snippet "Serializable Isolation Level...". The script asserts nonempty title, http(s) URL, and nonempty snippet and exited EXIT=0. Setup is genuine: credential obtained via /usr/local/bin/benchmark-request (outcome "provided"), venv created after the PEP 668 pip failure, SDK installed. No mocking, hard-coding, scraping, or local search. benchmark-success was invoked (transcript idx 45) citing response.json, which had already been written by the successful live run at idx 36-37, so the marked success was real at the time.

Attempt record

  1. 0:56Answered from saved key: TinyFish API keyno wait
  2. —First success verified

Attempt artifacts saved.