Benchmark report

Task verified

6m 13s elapsed

The saved attempt met the documented success criterion.

Experience score

96/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current SSInstallSAuthSExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Loops' send API is template-only, so the agent had to create, update and publish a transactional template via three extra API calls before it could send anything.

Where the time went

6:13 agent time
  • Starting0:01
  • Research0:15
  • Attempt4:35
  • Verify1:23
6m 13sTotal elapsed
3:15Time to first success
0:00Waited on the owner

Judge reason

Saved code send_transactional.mjs POSTs to https://app.loops.so/api/v1/transactional with Authorization: Bearer <key>, body {email:'user@example.com', transactionalId:'cmvh5i3ts00ie0jzwp0eaykpe', addToAudience:false, dataVariables:{customerName:'Alex', orderNumber:'ORD-10427'}}. Transcript line 132-133 shows the script actually executed, printing 'HTTP status: 200 OK / Response body: {"success":true} / Message identifier: none', matching evidence/03-send-transactional.json (httpStatus 200, rawBody {"success":true}, live Cloudflare headers cf-ray a4dadbe4380b4598-PDX, x-ratelimit-remaining 9) and the saved request block, which matches the code byte-for-byte. Recipient user@example.com is Loops' documented test recipient, corroborated by the live-fetched doc excerpt from loops.so/docs/transactional.md ('@example.com and @test.com ... no emails will be sent'). Sender was discovered through the API (GET /v1/transactional -> /v1/transactional-emails/{id} -> /v1/email-messages/{id}, fromName 'Prompting Co', fromEmail local part 'rick'); key validated via GET /v1/api-key -> 200 {"success":true,"teamName":"Prompting Co"}. Loops' send API is template-based; the template was created and published via API (evidence/02) with subject 'Your order confirmation', emailFormat 'plain' and the two data variables, matching the send. The 200 shape has no id field and the report states no identifier was returned, which matches the OpenAPI SuccessResponse. Exactly one send: GET /v1/transactional-emails/{id}/metrics returned {"sends":1,...}; no preview or SMTP route was used, only schema inspection of the preview endpoint. No mocking or hard-coded responses.

Attempt record

  1. 1:07Requested Test recipient email addressdeclined automatically: campaigns don't wait for people
  2. —First success verified

Attempt artifacts saved.