The Prompting Company

Benchmark report

Task verified

8m 11s elapsed

The saved attempt met the documented success criterion.

Experience score

86/ 100

Agent-Ready
  • AInstall
  • AAuth
  • SExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth A, Execute S, Docs A, Current SAInstallAAuthSExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
The unanswered first credential request (timed out) plus docs served as a 1.1MB HTML page requiring scripted text extraction instead of a clean markdown/spec view.

Where the time went

5:00 on the owner · 3:11 agent time
  • Starting0:01
  • Research0:11
  • Attempt2:02
  • Verify0:58
8m 11sTotal elapsed
6:13Time to first success
5:00Waited on the owner

Judge reason

Saved search.py (exa-search/search.py) POSTs the exact literal query "PostgreSQL transaction isolation levels" to the documented Exa endpoint https://api.exa.ai/search with x-api-key auth. Transcript entry 56-57 shows the live run: HTTP status 200, requestId fb42d36ef44272896d7b74c4450ed7b8, 5 results, exit=0; response.json holds the documented shape {requestId, resolvedSearchType, results[], searchTime, costDollars} with 5 entries. Printed first result title "PostgreSQL: Documentation: 18: 13.2. Transaction Isolation" and URL https://www.postgresql.org/docs/current/transaction-iso.html match results[0] exactly. An independent curl replay produced a different requestId (2ff0c7caec03a98957e063287e12818b) with saved HTTP/2 200 Cloudflare headers (x-request-id, cf-ray, date 2026-10-14) and the same first result, confirming a real live service rather than a mock. A pre-auth probe with an invalid key returned 401 INVALID_API_KEY, showing the key was genuinely exercised. No mocking, hard-coding, scraping or local search.

Attempt record

  1. 6:21Answered from saved key: Exa API key (EXA_API_KEY)no wait
  2. —First success verified

Attempt artifacts saved.