Benchmark report

Task verified

2m 43s elapsed

The saved attempt met the documented success criterion.

Experience score

100/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs S, Current SSInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Essentially nothing: first success at 46s after two doc fetches, one key request, and a single successful live run.

Where the time went

2:44 agent time
  • Research0:11
  • Attempt1:21
  • Verify1:11
2m 43sTotal elapsed
0:46Time to first success
0:00Waited on the owner

Judge reason

Saved code /workspace/home/exa-search/search.py POSTs to the documented endpoint https://api.exa.ai/search with x-api-key auth and body query="PostgreSQL transaction isolation levels", contents.highlights (per exa.ai/docs/reference/search). response.json records httpStatus 200 with a live Exa payload (requestId e3049c9c3fd45385188fd1882f8fc811, searchTime 1372.9, costDollars.search.neural 0.007) and 5 distinct results in the documented shape (id/title/url/text/highlights). First result: title "PostgreSQL: Documentation: 18: 13.2. Transaction Isolation", absolute URL https://www.postgresql.org/docs/current/transaction-iso.html, highlights[0] a 773-char nonempty excerpt; run-output.txt printed exactly those three values and the excerpt string matches response.json byte-for-byte. No mocking or hard-coding: the key came from benchmark-request and was read from a private file at runtime.

Attempt record

  1. 0:42Answered from saved key: EXA_API_KEYno wait
  2. —First success verified

Attempt artifacts saved.