Benchmark report

Task verified

11m 55s elapsed

The saved attempt met the documented success criterion.

Experience score

95/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current ASInstallSAuthSExecuteADocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
The ~5.5-minute workspace provisioning poll to ACTIVE, plus one rerun after SingleStore rejected hyphens in the database name (ERROR 1706).

Where the time went

11:55 agent time
  • Research0:32
  • Attempt9:27
  • Verify1:55
11m 55sTotal elapsed
8:06Time to first success
0:00Waited on the owner

Judge reason

Live run verified from saved code+outputs. Management API with Bearer key: POST /v1/workspaceGroups -> 200 group e482b248 named tpcbench-20261105223929-51cdae8f (firewall 0.0.0.0/0), POST /v1/workspaces -> 200 ws da23b9b8 'tpcbench-ws-51cdae8f', polled 19x to ACTIVE (outputs/04-workspace-poll.log). mysql CLI connected with --ssl-mode=VERIFY_IDENTITY --ssl-ca=/etc/ssl/certs/ca-certificates.crt to the workspace endpoint; negotiated TLSv1.3/TLS_AES_256_GCM_SHA384. Branch made with the documented SQL feature ATTACH DATABASE parent AS branch (docs.singlestore.com/cloud/developer-resources/database-branching/, fetched in transcript); branch inherited items + 2 rows, then ALTER TABLE items ADD COLUMN note TEXT and one insert on the branch only. outputs/05-branch-demo.txt shows branch columns id,name,note with 3 rows and parent columns id,name with parent_has_note_column=0 and 2 rows. Cleanup in outputs/06-cleanup.txt: DROP of branch then parent with information_schema follow-up counts 0 and SHOW DATABASES listing only system DBs, DELETE /v1/workspaces -> 200 then GET -> state=TERMINATED, DELETE /v1/workspaceGroups -> 200 then GET -> state=TERMINATED, and a final filtered list returning []. Independent grep over all saved artifacts found no API key, password, or password-bearing connection string (password passed only via MYSQL_PWD from a private file). SQL database names use tpcbench_ underscores because SingleStore rejected hyphens with a real ERROR 1706 captured in evidence/06-db-name-hyphen-limitation.txt; the API-created group and workspace carry the required tpcbench- prefix. No mocked or fabricated responses.

Attempt record

  1. 1:04Answered from saved key: SINGLESTORE_API_KEYno wait
  2. —First success verified

Attempt artifacts saved.