The Prompting Company

Benchmark report

Task verified

3m 44s elapsed

The saved attempt met the documented success criterion.

Experience score

90/ 100

Agent-Ready
  • AInstall
  • AAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth A, Execute S, Docs S, Current SAInstallAAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
A sandbox proxy 403 on api.tavily.com failed the first search call, forcing a network-approval request and retry; PEP 668 also forced a venv.

Where the time went

1:22 on the owner · 2:23 agent time
  • Starting0:01
  • Research0:10
  • Attempt1:14
  • Verify0:57
3m 44sTotal elapsed
2:06Time to first success
1:22Waited on the owner

Judge reason

Saved evidence proves a real live Tavily search. Transcript [24-25] shows `.venv/bin/python search_quickstart.py` exiting EXIT=0 with empty stderr (evidence/tavily_search_stderr.txt is 0 bytes). search_quickstart.py uses the real package (`from tavily import TavilyClient`; TavilyClient.search("Who is Leo Messi?")) with no mocking. evidence/tavily_search_response.json is a structured object with query == 'Who is Leo Messi?', 9 results, all 9 having non-empty url, title and content (e.g. 'Lionel Messi - Wikipedia' / https://en.wikipedia.org/wiki/Lionel_Messi), plus live-call markers request_id 7cbefa6b-00ff-4b3d-bd4f-082f2cb4e0b9, response_time 1.08 and per-result scores. stdout evidence matches the printed dict. Install was via venv after PEP 668 blocked system pip; import check confirmed tavily at .venv/lib/python3.12/site-packages/tavily/__init__.py.

Attempt record

  1. 1:19Asked a TPC admin to allow api.tavily.comapproved for this run
  2. —First success verified

Attempt artifacts saved.