The Prompting Company

Benchmark report

Blocked by a prerequisite

10m 0s elapsed

An external prerequisite prevented the documented task from completing. Benchmark again

Experience score

53/ 100

Critical Gating
  • SInstall
  • AAuth
  • DExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth A, Execute D, Docs S, Current SSInstallAAuthDExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
The sandbox proxy kept returning 403 on CONNECT to api.tavily.com after the allowlist approval reported confirmed:true, costing ~9 minutes of fruitless polling.

Where the time went

0:10 on the owner · 9:51 agent time
  • Starting0:01
  • Research0:10
  • Attempt8:21
  • Verify1:18
10m 0sTotal elapsed
—Time to first success
0:10Waited on the owner

Judge reason

No Tavily search ever executed. Setup is verified: transcript line 7 shows `pip install tavily-python` succeeding with `import tavily` resolving to .local/lib/python3.12/site-packages/tavily/__init__.py; the API key was obtained via benchmark-request (outcome provided) and tavily_quickstart.py matches the documented snippet (TavilyClient(api_key=...).search("Who is Leo Messi?")). Every execution failed at the network layer: saved tavily_error.txt shows requests.exceptions.ProxyError / OSError('Tunnel connection failed: 403 Forbidden') for api.tavily.com:443, and tavily_response.json is 0 bytes. A network approval for api.tavily.com returned {"outcome":"provided","confirmed":true}, yet 16 subsequent curl/SDK retries over ~9 minutes all returned 000 (CONNECT refused), while a same-moment control showed pypi.org code=200 and docs.tavily.com code=000, proving the proxy worked and the block was host-specific. The agent confirmed the SDK references only https://api.tavily.com, so no official alternative endpoint existed. The success criteria (printed response with query echo and non-empty results containing url/content) are unmet and benchmark-success was correctly never called.

Attempt record

  1. 0:56Asked a TPC admin to allow api.tavily.comapproved for this run
  2. 8:42Blocked by a prerequisiteSandbox proxy refused the HTTPS CONNECT tunnel to api.tavily.com:443 with 403 Forbidden even after the network request returned confirmed:true; proven host-specific by a pypi.org 200 control at the same moment.

Attempt artifacts saved.