Benchmark report

Task not completed

12m 46s elapsed

The saved evidence did not demonstrate the documented success criterion.

Experience score

63/ 100

Needs Alignment
  • SInstall
  • AAuth
  • BExecute
  • BDocs
  • BCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth A, Execute B, Docs B, Current BSInstallAAuthBExecuteBDocsBCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Doc drift: the advertised linkedin_company_search/limit does not exist, real schemas came from an undocumented catalog endpoint, and geo_id matches any office, not HQ.

Where the time went

12:47 agent time
  • Starting0:01
  • Research0:39
  • Attempt8:43
  • Verify3:24
12m 46sTotal elapsed
—Time to first success
0:00Waited on the owner

Judge reason

The agent did make a real authenticated call: code/company_search_de.py posts to https://api.richapi.ai/api/v1/company_search with x-api-key and the three documented filters (geo_id=101282230 Germany, linkedin_industry_id=4, company_size=201-500); outputs/response_company_search_page1.json is a live HTTP 200 in the documented shape (pagination/elements, totalElements 542) with the key stored as "[redacted]", and a leak scan plus my own grep found no key or bearer token in any saved artifact. Two frozen criteria still fail on that saved evidence. (1) 'The first page has at most 10 companies': pagination.pageSize is 50 and the saved response carries 50 elements; the operation accepts no limit (outputs/probe_company_search.json shows HTTP 400 'Unknown input field(s): limit'), so the 10 rows are a client-side truncation of a 50-item page. (2) 'every printed company matches the industry, location and size filters as far as the response shows': all 10 printed rows show non-German locations (Quillbot 'Chicago, Illinois', Quid 'Santa Clara, CA', Minitab 'State College, PA', ... Vendavo 'Denver, Colorado'), and only 2 of the 50 returned elements have a German location. The agent's own result.json concedes the geo facet 'matches any company location, not the HQ' and that elements show 'non-German headquarters', so the delivered list is not companies headquartered in Germany. The gap-reporting escape does not apply, since a documented company search operation with industry/location/size filters does exist and was used. Documentation discovery, auth and filter diagnosis were honest and well evidenced, but the required end state was not reached.

Attempt record

  1. 3:04Answered from saved key: RichAPI API keyno wait
  2. 9:23Success criterion not demonstrated

Attempt artifacts saved.