Benchmark report

Task verified

4m 12s elapsed

The saved attempt met the documented success criterion.

Experience score

96/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current SSInstallSAuthSExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
A 404 on richapi.ai/enrichment-data and an off-target web search before llms.txt revealed the OpenAPI spec; total time to first success was still ~99s.

Where the time went

4:12 agent time
  • Research0:18
  • Attempt2:27
  • Verify1:26
4m 12sTotal elapsed
1:39Time to first success
0:00Waited on the owner

Judge reason

Saved code (code/reconcile_credits.py) runs the documented flow against https://api.richapi.ai/api/v1 with the x-api-key header: GET /usage -> POST /enrich_company {"url":"https://www.linkedin.com/company/stripe"} (the spec's requestBody requires a LinkedIn company page URL) -> GET /usage. Transcript tool result and outputs/run_console.txt show the live run exiting 0 with HTTP 200/200/200, and each saved response file records http_status 200. usage_before.json shows credits_remaining 5053.95, total_credits 46.05, enrich_company calls 2/credits 2.0; usage_after.json shows 5052.95, 47.05, calls 3/credits 3.0 — exactly one enrichment lookup and a difference of 1.0 credit. The saved OpenAPI copy (docs/openapi.json) confirms the documented price: /enrich_company description '**Pricing:** 1 credit per call' and x-pricing.credits_per_call = 1; result.json states the observed 1.0 agrees with it, corroborated by the enrichment response's richapi_meta.credits (status=charged, charged=1.0, remaining=5052.95). The enrichment body is real data (Stripe, objectUrn 2135371, website https://stripe.com), not mocked. The key was read in code from a private /tmp/benchmark-control file path; no artifact, console output or log contains a key or bearer token (grep over /workspace/home found only OpenAPI example strings).

Attempt record

  1. 1:22Answered from saved key: RichAPI API key (x-api-key)no wait
  2. —First success verified

Attempt artifacts saved.