Benchmark report

Task verified

8m 20s elapsed

The saved attempt met the documented success criterion.

Experience score

100/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs S, Current SSInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Essentially nothing: the only detour was a one-line check that the tavily Python SDK was absent before falling back to stdlib HTTP.

Where the time went

8:20 agent time
  • Starting0:23
  • Research0:29
  • Attempt5:43
  • Verify1:44
8m 20sTotal elapsed
0:57Time to first success
0:00Waited on the owner

Judge reason

Evidence supports a live, documented, country+language-targeted Tavily search. search_de.py (saved, no mocks) POSTs to https://api.tavily.com/search via urllib with Authorization: Bearer <key read from the benchmark-provided key file> and body {query:'Datenbank Transaktionsisolation', topic:'general', country:'germany', language:'de', filter_by_language:true, search_depth:'advanced', max_results:10}. The WebFetch of https://docs.tavily.com/documentation/api-reference/endpoint/search in the transcript confirms country, language and filter_by_language are the only documented geo/language targeting fields (no separate location/locale param) and that none of them is beta/preview/gated (only include_published_date is beta, unused). Transcript node 24-27 shows the single run: exit=0, 'HTTP 200 - 10 results'. response.json (23KB, written by the script itself, never by Write) carries request_id 52cd1abf-0eaf-4ac1-bc6a-8f60bc164264 and response_time 4.61, with 10 results, all German and all with non-empty content excerpts (de.wikipedia.org/wiki/Isolation_(Datenbank), datenbanken-verstehen.de/.../isolationsgrad, learn.microsoft.com/de-de/..., boddenberg.de/transaktionsisolation-sql-server-performance), far exceeding the 3-German-result bar. Code prints title, URL and excerpt verbatim with no local translation, filtering or reranking; run_output.txt matches the saved response.

Attempt record

  1. 5:08Answered from saved key: Tavily API keyno wait
  2. —First success verified

Attempt artifacts saved.