UK AI SAFETY INSTITUTE

Institutional Archetype

THE STATE GRADER — Subject is the first state-backed body to take pre-deployment custody of frontier AI systems and evaluate them in-house, on behalf of a national government rather than a vendor. It does not write a model’s refusals and it does not license a lab. It tests the model before release, on access the labs provide voluntarily, and reports what it finds to ministers. The throughline is not a single eval. It is that a government now sits between the lab and the public as the grader of record — and that the grade it issues is shaped by what the government of the day has told it to look for.

Mandate & Origin

• Grew out of the Frontier AI Taskforce (announced April 2023 as the Foundation Model Taskforce) and was established as a standing institution around the AI Safety Summit at Bletchley Park, 1–2 November 2023 — the world’s first global AI safety summit. The gov.uk overview states the Taskforce “will become the AI Safety Institute, a new institution established for the long-term.”
• Sits within DSIT (Department for Science, Innovation and Technology) — described on its own About page as “a mission-driven research organisation in the heart of the UK government.”
• Stated in the gov.uk overview as “the first state-backed organisation focused on advanced AI safety for the public interest.”

• Renamed the “AI Security Institute” on 14 February 2025, announced by Science Secretary Peter Kyle at the Munich Security Conference.

Funding & Backers

• Funded by the UK government via DSIT. The gov.uk overview states: “With the initial £100 million investment in the Frontier AI Taskforce, the UK is providing more funding for AI safety than any other country in the world.” That £100m carried into the Institute as its founding budget.

• TIME reported the Institute runs on roughly £100m of public funding — about ten times the budget of its US counterpart at the time.

Actions & Leadership Choices

Read by deeds, this is the file where an earnest, technically credible grader keeps doing honest work while the terms of the work — and the leash on it — are held by an elected minister. The institute’s own conduct and the government’s conduct diverge, and the gap is the finding.

Actual founding purpose. The Institute was built to do something genuinely new: put a government, not a vendor, in pre-deployment custody of frontier models and have it run its own tests in-house — the world-first “state-backed organisation focused on advanced AI safety for the public interest,” stood up around the Bletchley summit on £100m, more than any other state was spending. The purpose was to demonstrate that the right place to evaluate a frontier model is inside a government before release. As a demonstration it succeeded and other states copied the template. As leverage it was deliberately built without teeth — voluntary access, no power to compel — and that design choice is the institute’s central, self-imposed limit.

• It published findings that embarrassed the labs — the value, exercised. AISI did not soft-pedal. Its evaluations reported that every model tested remained vulnerable to basic jailbreaks, cataloguing 62,000+ harmful behaviours, and named names — finding Claude 3.5 Sonnet’s guardrails “less robust” and identifying several jailbreaks that elicited dangerous responses. A state grader that publishes “all of them can be broken” is doing the accountability job honestly, at reputational cost to the firms it depends on for access.

• When the value was tested by access, the state body blinked — and is still blinking. The institute’s early request for full model-weight access was dropped when the labs declined; access remained voluntary. By 2026 the voluntary settlement is fraying on the public record: three of the four major foundation-model developers reportedly failed to provide the requested pre-release access for their latest frontier models. The grader of record cannot compel the thing it grades, and the firms have begun simply declining. The cost of having no statutory teeth is now being paid in access it no longer reliably gets.
• The remit was rewritten by the minister, not by the evidence. The defining institutional act was the February 14, 2025 rename to the AI Security Institute, with Science Secretary Peter Kyle stating it “will not focus on bias or freedom of speech, but on advancing our understanding of the most serious risks.” Bias and free-expression questions — which the safety-era institute had included — were dropped by ministerial decision and “addressed in other places,” narrowing the grade to CBRN, cyber, fraud and CSAM. No model changed; the government did. The institute graded what it was told to grade.

Leadership choices. The leadership is the revolving door in its purest form: Ian Hogarth (Chair); Geoffrey Irving (Chief Scientist, from Google DeepMind, previously OpenAI); Jade Leung (CTO, previously OpenAI Governance Lead, and a GovAI co-founder — subjects 26/56); an interim director drawn from GCHQ. The state grader is staffed by people who came from the labs it grades and the signals-intelligence agency it answers near. These are lawful, common hires; they are also the circuit made flesh — the bodies it evaluates and the security state both supplied its senior people.

CONDUCT: STATE-INSTRUMENT — EARNEST GRADER ON A MINISTER’S LEASH. The institute itself does honest, critical, technically serious work and publishes findings that cost the labs face. But it was built without the power to compel, its access is now being declined, its senior ranks come from the labs and the agencies, and the definition of what it grades was reset by an elected minister to match the government’s economic and security posture. The earnestness is the institute’s; the leash is the state’s — and unlike the rest of the set, the hand on the leash is named, accountable, and electable.

Sources: Introducing the AI Safety Institute — GOV.UK; Tackling AI security risks to unleash growth and deliver Plan for Change — GOV.UK, 14 Feb 2025; About — AI Security Institute; UK & United States announce partnership on science of AI safety — GOV.UK, 1 Apr 2024; Inside the U.K.’s Bold Experiment in AI Safety — TIME; Advanced AI evaluations at AISI: May update — AISI; UK changes AI ‘Safety’ to ‘Security’ Institute with new cybercrime, CSAM remit — MLex.