The Prompting Company

Benchmark report

Blocked by a prerequisite

19m 10s elapsed

An external prerequisite prevented the documented task from completing. Benchmark again

Experience score

53/ 100

Critical Gating
  • SInstall
  • AAuth
  • DExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth A, Execute D, Docs S, Current SSInstallAAuthDExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
The egress proxy's 403 on CONNECT api.tavily.com cost ~16 minutes of polling and two approval round-trips that reported confirmed but never unblocked the host.

Where the time went

0:19 on the owner · 18:52 agent time
  • Starting0:01
  • Research0:15
  • Attempt17:30
  • Verify1:06
19m 10sTotal elapsed
—Time to first success
0:19Waited on the owner

Judge reason

Setup is proven: venv + `.venv/bin/pip install tavily-python` succeeded (sdk_version.txt: tavily-python 0.8.4; import resolved to .venv/.../tavily/__init__.py), and the API key was supplied via benchmark-request (outcome provided, VALUE file) and read at runtime by search_messi.py using TavilyClient(api_key=...).search("Who is Leo Messi?"). The call never reached Tavily: the saved run exited 1 with requests.exceptions.ProxyError / 'Tunnel connection failed: 403 Forbidden' (evidence/tavily_search_stderr.txt), and evidence/tavily_search_response.json is 0 bytes. Verbose curl shows 'CONNECT api.tavily.com:443' answered 'HTTP/1.1 403 Forbidden' by the sandbox egress proxy, while the same proxy returns pypi=200 and served the pip install; api/mcp/root/docs tavily.com hosts all returned 000. Two benchmark-request network approvals for api.tavily.com both replied {"outcome":"provided","confirmed":true} yet ~30 retries over 21:34-21:50 UTC stayed 403. No HTTP 200, no query field, no results list, so the success criteria are unmet; benchmark-first-success.json does not exist and benchmark-success was correctly not run.

Attempt record

  1. 7:00Asked a TPC admin to allow api.tavily.comapproved for this run
  2. 18:05Blocked by a prerequisiteSandbox egress proxy refuses CONNECT to api.tavily.com:443 with HTTP 403 despite two confirmed hostname approvals, so the documented Tavily search endpoint is unreachable from the run environment.

Attempt artifacts saved.