Benchmark report

Task verified

5m 33s elapsed

The saved attempt met the documented success criterion.

Experience score

94/ 100

Agent-Ready
  • AInstall
  • SAuth
  • SExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth S, Execute S, Docs A, Current SAInstallSAuthSExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Loops' template-based send forced a create/update-LMX/publish prerequisite chain, and no documented test recipient or sandbox meant extra doc searching and a skipped input request.

Where the time went

5:33 agent time
  • Starting0:01
  • Research0:15
  • Attempt3:54
  • Verify1:23
5m 33sTotal elapsed
2:48Time to first success
0:00Waited on the owner

Judge reason

Live, authenticated send verified. Transcript line 112 shows `node send.mjs` printing `HTTP status: 200 / Response body: {"success":true}` against POST https://app.loops.so/api/v1/transactional (docs loops.so/docs/api-reference/send-transactional-email), with the key obtained via benchmark-request and validated by GET /v1/api-key -> 200 {"success":true,"teamName":"Prompting Co"} (evidence/01). Loops' send API is template-based, so the agent created/updated/published the template through the API (evidence/04: 201, 200, 200) with subject 'Loops API transactional test' and a plain-text LMX body; the saved request in evidence/05_send.json ({email, transactionalId cmvh4ebeg01o30jz3l0767kr2, addToAudience:false, dataVariables customerName/orderNumber}) matches send.mjs exactly, and the printed status matches the saved response. The report correctly states the API returns no message identifier, consistent with the OpenAPI 200 schema. Sender was discovered via the API (GET /v1/email-messages/... -> fromName 'Prompting Co', fromEmail 'rick'), not guessed. Loops documents no test recipient/sandbox (agent checked docs, llms.txt, /agents); the benchmark input request for a recipient returned 'skipped', so it used a null-MX example.com address and confirmed deliveries:0 via the metrics endpoint. Two deviations, both disclosed and non-fatal: send.mjs was accidentally re-run once while capturing console output (metrics show sends:2, both identical 200s to the same no-mailbox address), and evidence/06 is a hand-transcribed copy of the run output at transcript line 112 rather than a captured stream. No mocking or hard-coded response.

Attempt record

  1. 1:17Requested Test recipient email addressdeclined automatically: campaigns don't wait for people
  2. —First success verified

Attempt artifacts saved.