evilrobots.lol / tech · evaluator record

Evaluator Record

One record per evaluator: what each was engaged to test, on what terms, what it found in the document's own words, and what happened afterwards, with the money and seats behind it. Every line is a store record and every record links to the document it was read from.

A lab’s safety claim names an outside tester, and the tester’s report names its terms, or does not. These pages keep the ledger per evaluator: every engagement a lab document records, the access window, model stage, publication right and payment that document states, the finding quoted, and the events on the record afterwards, from an incident on a model it tested to a lab’s statement that it acted or did not act on what it found. A term the document is silent on reads not stated, and each table counts how many of its rows state each field. The records are drawn from the same store as the Evaluator Observatory; the narrative is on The Evaluator Alliances.

▣ SLICE 38 / 38 evaluators
Lab
Type
year, month or day

Filters combine: any of the values ticked within a dimension, all of the dimensions at once. A table that has no column for a dimension you have set says so instead of showing rows that cannot match. The address bar carries the slice.

The records 38 evaluators, store as of 2026-10-18

Ordered by engagements on record, then outcomes, then money rows. Open a record for the timeline, the terms grid, the findings quoted and what followed; a lab name narrows the list to the evaluators its documents name.

EvaluatorTypeEngagementsLabsOutcomesMoney rowsSeatsMovesDossier
UK AISIUK AI Security Institutegovernment15OpenAI, Anthropic, Google DeepMind, Microsoft5594dossier
SecureBiononprofit14OpenAI, Anthropic, Meta21930dossier
Gray Swanfor-profit12OpenAI, Anthropic, Google DeepMind, Meta2330dossier
Irregularfor-profit10OpenAI, Anthropic, Google DeepMind, Meta4420dossier
US CAISICAISI (NIST)government10OpenAI, Anthropic, Google DeepMind2033dossier
Redwood Researchnonprofit8Anthropic1863dossier
METRnonprofit7OpenAI, Anthropic519156dossier
Apollo Researchnonprofit7OpenAI, Anthropic, Google DeepMind, Meta4821dossier
Signature Sciencenonprofit7Anthropic0000dossier
Trajectory Labsfor-profit7Anthropic, Google DeepMind, Meta0000dossier
Andon Labsfor-profit5Anthropic3020dossier
Facultyfor-profit5OpenAI, Anthropic, Google DeepMind, Meta1310dossier
Gryphon Scientificfor-profit4OpenAI, Anthropic0200dossier
10a Labsfor-profit4Anthropic0000dossier
Vals AIfor-profit3Anthropic, Mistral AI0700dossier
Epoch AInonprofit2OpenAI, Anthropic32464dossier
Scale AIScale AI (SEAL)for-profit2Anthropic, Meta1021dossier
MERIDIAN LABSnonprofit2Anthropic0233dossier
Artificial Analysisfor-profit1Mistral AI0600dossier
ARCALIGNMENT RESEARCH CENTER (ARC)nonprofit0none on record2554dossier
MECHANIZEfor-profit0none on record1632dossier
FAR.AInonprofit0none on record01584dossier
Stanford CRFMacademic0none on record0910dossier
AVERInonprofit0none on record0840dossier
Collective Intelligence ProjectCollective Intelligence Project (CIP)nonprofit0none on record0800dossier
Princeton Holistic Agent LeaderboardPrinceton Holistic Agent Leaderboard (HAL)academic0none on record0800dossier
ArenaArena (formerly LMArena, Chatbot Arena)for-profit0none on record0700dossier
Translucenonprofit0none on record0782dossier
Palisade Researchnonprofit0none on record0672dossier
AIUCfor-profit0none on record0510dossier
Dreadnodefor-profit0none on record0400dossier
GENERALITY LABSunstated0none on record0320dossier
Humane Intelligencenonprofit0none on record0313dossier
Trail of Bitsfor-profit0none on record0210dossier
Apart Researchnonprofit0none on record0010dossier
HiddenLayerfor-profit0none on record0000dossier
Intl. AISI NetworkInternational Network for Advanced AI Measurement, Evaluation and Science (formerly the International Network of AI Safety Institutes)government0none on record0000dossier
PROMPTFOOfor-profit0none on record0000dossier

Dated 125 of 125Model named 117 of 125Terms stated 66 of 125Finding quoted 53 of 125

Access window 21 of 125Model stage 26 of 125Publication 11 of 125Payment 6 of 125Finding, quoted 53 of 125

Outcomes: Dated 18 of 18Outcomes: Model named 12 of 18Outcomes: Tied to an engagement 13 of 18Outcomes: Evaluator named by the record 18 of 18