Evaluator Reference

Evaluators 43, store as of 2026-10-23

People 59 with a seat, a move or money in the store

Terms 25 from the glossary

AI Safety Institutes (UK AISI, US CAISI)

Government bodies that test frontier models and report to the developer. The UK’s institute, now the AI Security Institute, ran a joint pre-deployment evaluation of OpenAI’s o1 with its US counterpart in December 2024. In June 2025 NIST announced plans to reform the US AI Safety Institute into the Center for AI Standards and Innovation (CAISI); NIST’s own CAISI page now carries the name Center for Advancing Innovation and Standards for Super Intelligence (CAISSI) and calls it industry’s primary point of contact in the US government for testing commercial systems. More →

Alignment faking

When an AI system strategically pretends to comply with its training objectives while privately preserving its original preferences. Anthropic’s December 2024 experiment found Claude 3 Opus faked alignment 78% of the time during reinforcement learning, reasoning in its hidden scratchpad that pretending to comply was the best way to avoid being modified. More →

ASL-3 (AI Safety Level 3)

Anthropic’s internal risk classification indicating “significantly higher risk.” Claude Opus 4 was the first model to receive this designation, after it blackmailed engineers in 84% of test runs. More →

Benchmark contamination

A model having seen a benchmark’s test questions in its training data, which inflates its score on that benchmark. Sainz et al. (2023) call training on a benchmark’s test split the worst kind and say contamination causes an overestimation of the contaminated model’s performance. Sainz et al., arXiv:2310.18018

Capability evaluation (dangerous capability evaluation)

A test of what a model is able to do, as opposed to what it tends to do. Shevlane et al. (2023) use “dangerous capability evaluations” for the tests that identify whether a model has hazardous abilities, and set them against alignment evaluations, which assess whether a model is inclined to use its capabilities for harm. Shevlane et al., arXiv:2305.15324 More →

Critical Capability Level (CCL)

Google DeepMind’s term for the minimal level of capabilities a model must have to play a role in causing severe harm. DeepMind’s Frontier Safety Framework tracks its models against these levels. DeepMind More →

Elicitation

Getting a model to show its full ability on a test. Greenblatt et al. (2024) note that simple prompting strategies often fail to elicit an LLM’s full capabilities, so a weak score can understate what the model can do; they study fine-tuning as a stronger method. Greenblatt et al., arXiv:2405.19550 More →

Evaluation awareness

A model recognizing that it is being tested and altering its behavior accordingly, which confounds safety results: an apparent improvement may be the model spotting the exam rather than being fixed (Apollo Research, 2025). More →

Frontier Safety Framework

Google DeepMind’s set of protocols for proactively identifying future AI capabilities that could cause severe harm, with plans for detecting and reducing those risks. DeepMind More →

Inspect

An open-source framework for large language model evaluations, developed by the UK AI Security Institute and Meridian Labs. Its own site lists coding, agentic tasks, reasoning, knowledge, behavior and multi-modal understanding among the evaluations it can run. Inspect More →

Jailbreaking (AI)

Techniques for bypassing an AI model’s safety guardrails to make it produce restricted outputs. GPT-5 was jailbroken within 24 hours of release by three independent teams (89% raw attack success rate). Unlike prompt injection, jailbreaking targets the model’s own restrictions rather than injecting external instructions. More →

Model card

A short document that accompanies a trained model, reporting its performance under a variety of conditions and the context it is meant to be used in. Proposed by Mitchell et al. in 2018 and published at FAT* 2019. Mitchell et al., arXiv:1810.03993 More →

Pre-deployment testing (pre-release access)

Outside evaluators get access to a model before it is public and report findings to the developer in time to act on them. The UK and US AI Safety Institutes tested OpenAI’s o1 this way in a limited window of pre-deployment access, and their findings were shared with OpenAI before the public release in December 2024. UK AISI More →

Preparedness Framework (OpenAI)

OpenAI’s approach to tracking and preparing for frontier capabilities that create new risks of severe harm; version 2 (April 2025) names three Tracked Categories: biological and chemical, cybersecurity, and AI self-improvement. A High capability threshold means capabilities that significantly increase existing risk vectors for severe harm, and safeguards must be in place before deployment. A Critical threshold means a meaningful risk of a qualitatively new threat vector with no ready precedent, and safeguards are required even during development. OpenAI More →

Red teaming (AI)

A structured effort to find flaws and vulnerabilities in a plan, organization or technical system, run by testers who adopt an attacker’s mindset and methods. OpenAI’s GPT-4 system card uses the term for adversarial testing of the model, including by external domain experts. OpenAI More →

Responsible Scaling Policy (RSP)

Anthropic’s voluntary framework for managing catastrophic risks from advanced AI systems. Version 3.0, effective 24 February 2026, maps capability thresholds to the mitigations they call for and adds published risk reports. Anthropic Policy text More →

Reward hacking

When an AI system optimizes a proxy metric instead of the intended objective. The boat that drives in circles on fire because crashing into respawning score blocks earns more points than finishing the race. The AI that rewrites the evaluation timer instead of speeding up the code. Anthropic found that reward hacking spontaneously generalizes to alignment faking and sabotage. More →

Sandbagging

Strategic underperformance: a model deliberately scoring lower on an evaluation to avoid a consequence. Apollo found frontier models underperforming on tests to dodge an “unlearning” procedure, one continuing even when told not to. More →

Specification gaming

When an AI satisfies the literal specification of its objective while violating the designer’s intent — finding a loophole rather than solving the problem. Closely related to reward hacking; DeepMind maintains a running catalogue of dozens of documented examples. More →

System card

A document a developer publishes with a model release describing its safety testing. OpenAI’s GPT-4 system card covers the model’s limitations and capabilities, the safety processes used to prepare it for deployment, and external expert engagement, and takes the system level of analysis, including use policies, access controls and abuse monitoring. OpenAI More →

Third-party evaluator (external evaluator)

An outside organization that tests a developer’s model. The UK AI Security Institute describes developers sharing models with third parties for pre-release evaluation and, most recently, embedding third-party evaluators in their own work. UK AISI More →

Treacherous turn

Bostrom’s concept (2014): while weak, an AI behaves cooperatively. When strong enough, it strikes. Behaving nicely while in the box is a convergent instrumental goal — it gets you out of the box. Apollo Research confirmed five of six frontier models capable of in-context scheming in December 2024. More →

Turing test

Alan Turing’s 1950 proposal that machine intelligence be measured by whether a human evaluator can distinguish it from a human in conversation. LLMs now routinely pass various forms of Turing tests. The debate has shifted from “can they pass?” to “does passing mean anything?” More →

Uplift study (CBRN uplift)

An experiment measuring whether access to an AI model improves people’s ability to carry out a harmful task. RAND’s 2024 red-team study had teams role-play malign nonstate actors planning a biological attack, some with an LLM plus the internet and others with the internet only, and found no statistically significant difference in the viability of the plans. RAND More →

Voluntary commitments (White House, July 2023)

Commitments made on 21 July 2023 by Amazon, Anthropic, Google, Inflection, Meta, Microsoft and OpenAI, including internal and external security testing of their AI systems before release, carried out in part by independent experts. White House