Benchmark report

Onboarding verified

2m 23s elapsed

The saved attempt met the documented success criterion.

Experience score

98/ 100

Agent-Ready
  • AInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth S, Execute S, Docs S, Current SAInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
The system Python being PEP 668 externally-managed blocked 'pip install polars', costing one extra step to create a venv; nothing in Polars itself slowed the run.

Where the time went

2:23 agent time
  • Starting0:01
  • Research0:11
  • Attempt1:07
  • Verify1:05
2m 23sTotal elapsed
0:36Time to first success
0:00Waited on the owner

Judge reason

Saved evidence proves real execution of Polars 1.44.2 (installed into .venv after system pip refused with PEP 668; wheels polars-1.44.2-py3-none-any.whl + polars_runtime_32 aarch64 downloaded). Transcript entry 16 ran '.venv/bin/python quickstart.py | tee evidence/stdout.txt' and the result shows 'EXIT=0'. evidence/stdout.txt matches: original 'shape: (4, 4)' table with Alice Archer/Ben Brown/Chloe Cooper/Daniel Donovan and dtype row str/date/f64/f64; re-read table from output.csv identical with Schema({'name': String, 'birthdate': Date, 'weight': Float64, 'height': Float64}) and 'Round-trip equals original: True' from df_csv.equals(df); group_by decade output 'shape: (2, 4)' with columns decade/n/avg_height/avg_weight (1980 -> 3, 1.72, 69.73; 1990 -> 1, 1.56, 57.9) — fewer than 4 rows, and the aggregates recompute correctly (5.17/3=1.72, 209.2/3=69.73). output.csv exists on disk (164 bytes) with the four expected rows. Polars is a local library, so no credentials or mocks are involved; the package itself did the work.

Attempt record

  1. —First success verified

Attempt artifacts saved.