Benchmark report

Blocked by a prerequisite

8m 47s elapsed

An external prerequisite prevented the documented task from completing. Benchmark again

Experience score

46/ 100

Critical Gating
  • AInstall
  • AAuth
  • DExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth A, Execute D, Docs A, Current SAInstallAAuthDExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
The sandbox egress proxy returned 403 'Destination not approved' for all six Tavily hosts, and the 5-minute egress-allowlist request timed out with no response.

Where the time went

5:09 on the owner · 3:39 agent time
  • Starting0:01
  • Research0:11
  • Attempt2:24
  • Verify1:03
8m 47sTotal elapsed
—Time to first success
5:09Waited on the owner

Judge reason

Setup completed and is verifiable: pip3 install hit PEP 668, so the agent created /tmp/benchmark-home/.venv and installed tavily-python 0.8.4 + requests 2.34.2 (evidence/installed_packages.txt; 'tavily-python OK' import check at transcript line 15), obtained an API key via benchmark-request (outcome 'provided', line 18), and wrote tavily_quickstart.py using TavilyClient(api_key=...).search("Who is Leo Messi?"). The evaluated call never executed: tavily_search_response.json is 0 bytes and tavily_search_stderr.txt shows requests.exceptions.ProxyError -> OSError('Tunnel connection failed: 403 Forbidden') for api.tavily.com:443 on both the initial run and the 11:19:45Z retry. Cause is environmental and proven, not an agent error: evidence/egress_probe.json shows raw CONNECT to the sandbox proxy returning 'HTTP/1.1 403 Forbidden ... Destination not approved' for api.tavily.com:443/:80, tavily.com, docs.tavily.com, app.tavily.com and mcp.tavily.com; the documented cURL alternative failed with 'curl: (56) CONNECT tunnel failed, response 403'; and a non-proxied direct connect to 54.210.210.249:443 timed out. The agent requested egress approval via benchmark-request and got outcome 'timed_out'. No mock or fabricated response was produced and benchmark-success was never called, so no non-empty 'results' array with title/url/content was ever printed.

Attempt record

  1. 1:54Asked the owner to act: Allow network egress to api.tavily.comno response within the window
  2. 7:44Blocked by a prerequisiteSandbox egress is denied to every Tavily host (proxy CONNECT returns 403 'Destination not approved' for api.tavily.com:443/:80, tavily.com, docs/app/mcp.tavily.com; direct TCP 443 times out), so the documented api.tavily.com call cannot reach the service; the egress-allowlist request timed out.

Attempt artifacts saved.