The Prompting Company

Benchmark report

Task verified

12m 25s elapsed

The saved attempt met the documented success criterion.

Experience score

90/ 100

Agent-Ready
  • AInstall
  • AAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth A, Execute S, Docs S, Current SAInstallAAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Two timed-out credential requests (~10 min to first success) plus a detour into the agent-identity/auth.md key-minting flow, and the PEP 668 pip failure.

Where the time went

8:00 on the owner · 4:26 agent time
  • Starting0:01
  • Research0:11
  • Attempt2:58
  • Verify1:16
12m 25sTotal elapsed
10:04Time to first success
8:00Waited on the owner

Judge reason

Saved code parallel-search/search.py POSTs {"objective":"PostgreSQL transaction isolation levels","search_queries":["PostgreSQL transaction isolation levels"]} to the documented endpoint https://api.parallel.ai/v1/search with x-api-key. Transcript tool result (live run at 22:32:19Z with the key file issued by benchmark-request, outcome 'provided') printed 'HTTP status: 200', search_id search_6dca1d9355d2c17c36a8940fc15f2a49, result count: 10, first result title 'Documentation: 18: 13.2. Transaction Isolation' and url https://www.postgresql.org/docs/current/transaction-iso.html. Those exact values appear in evidence/search_response.json with the documented shape (search_id, results[], usage, session_id; items have url/title/publish_date/excerpts) containing 10 distinct real results with substantive excerpts. An independent second live call through the official parallel-web 1.3.5 SDK (client.search) returned a different search_id (search_a29991e77a273f75ed95726f0d0f53e7) with the same top result, and an unauthenticated probe returned HTTP 401 'Invalid API key (C.1)' — confirming real authenticated network calls, not mocks or hard-coding.

Attempt record

  1. 10:08Answered from saved key: PARALLEL_API_KEYno wait
  2. —First success verified

Attempt artifacts saved.