The Prompting Company

Benchmark report

Blocked by a prerequisite

17m 39s elapsed

An external prerequisite prevented the documented task from completing. Benchmark again

Experience score

50/ 100

Critical Gating
  • AInstall
  • AAuth
  • DExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth A, Execute D, Docs S, Current SAInstallAAuthDExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Sandbox credential-proxy 502s on api.tavily.com consumed ~12 minutes of backoff retries across SDK and REST paths, never yielding a response.

Where the time went

3:40 on the owner · 13:59 agent time
  • Starting0:01
  • Research0:12
  • Attempt12:46
  • Verify1:01
17m 39sTotal elapsed
—Time to first success
3:40Waited on the owner

Judge reason

Setup completed and is verified: venv created after pip hit PEP 668, tavily-python installed and importable (transcript line 11: 'tavily ok .../site-packages/tavily/__init__.py'), API key obtained via benchmark-request (outcome=provided), quickstart script written exactly per docs (TavilyClient(api_key).search("Who is Leo Messi?")), and network approval granted for api.tavily.com (confirmed=true). But no live search response was ever obtained: the first run died with ProxyError 403 tunnel, and after approval every call returned HTTP 502 with text/plain body 'Credential proxy unavailable' and no Tavily-origin headers (probe.py output, tavily_error.txt SDK traceback, retry_log.txt showing 28 logged attempts 06:32:10-06:40:34 plus earlier ones, 37 total over ~12 min). tavily_response.json is 0 bytes, no results array or query echo exists anywhere in the evidence, benchmark-success was not called, and the agent itself reports status=blocked/success=false. Same-proxy control egress succeeded (https://pypi.org/simple/ -> 200), isolating the fault to the sandbox credential-proxy path for api.tavily.com rather than Tavily, the SDK, or the key.

Attempt record

  1. 2:04Asked a TPC admin to allow api.tavily.comapproved for this run
  2. 16:38Blocked by a prerequisiteSandbox egress credential proxy returned HTTP 502 'Credential proxy unavailable' for api.tavily.com on all 37 attempts over ~12 minutes, so no request could reach Tavily despite an approved host and a user-provided API key.

Attempt artifacts saved.