CENTER FOR AI SAFETY

Institutional Archetype

THE EXTINCTION STATEMENT — CAIS is the small nonprofit whose largest export is a frame. It runs a research program — hazardous-knowledge benchmarks, unlearning methods, a frontier-difficulty exam — but its civilizational footprint is a twenty-two-word statement it organized in May 2023 and the existential register that statement installed in the policy conversation. The throughline is not headcount or budget; both are modest against the labs it convenes. The throughline is that when the most senior people in frontier AI wanted to co-sign that the technology might end the species, CAIS is the body that hosted the page they signed.

Mandate & Origin

• Founded 2022; described in third-party records as co-founded by Dan Hendrycks (its Executive & Research Director) and Oliver Zhang. CAIS’s own About page states only its mission and does not name a founding year or founders.

• A 501(c)(3) nonprofit based in San Francisco, self-described on its own site simply as “an AI safety non-profit.” 501(c)(3) status and SF location are confirmed in nonprofit-registry aggregators rather than an IRS record opened directly.
• Stated mission, verbatim from CAIS’s own About page: “Our mission is to reduce societal-scale risks from artificial intelligence.”
• A separate Center for AI Safety Action Fund — a 501(c)(4) advocacy arm formed July 2023 — carries the lobbying that a 501(c)(3) cannot, and was a named sponsor of California’s frontier-model bill SB 1047.

Funding & Backers

• Received $6.5 million from the FTX Future Fund in 2022; after FTX collapsed, the bankruptcy estate sought to recover the money. Bloomberg reported the clawback probe (Oct 25 2023).

• A major recurring funder is Open Philanthropy (general-support and fellowship grants across 2022–2023; now operating as Coefficient Giving). Exact recent dollar figures could not be opened directly on Open Phil’s own redesigned pages and are omitted rather than reported at a precision the source does not support.
• Jaan Tallinn is a funder and a CAIS board member — a documented adjacency to the same Estonian financier who backs the broader existential-risk network.

• The funding shape is the on-thesis tension: a body warning that AI is a societal-scale risk was seed-funded by crypto-adjacent and EA-adjacent existential-risk money, not by the public it speaks for.

Actions & Leadership Choices

Founding purpose, judged on evidence. CAIS was founded in 2022 by Dan Hendrycks and Oliver Zhang as a 501(c)(3) “to reduce societal-scale risks from artificial intelligence.” On the deed record, the purpose it actually pursued is narrower and sharper than the mission line: CAIS was built to install the existential frame as the governing altitude for AI policy and to build the evaluation apparatus — hazardous-knowledge benchmarks, a frontier exam — that gives that frame operational teeth. That is a real and coherent purpose, not a cover; the extinction statement and the WMDP/Humanity’s-Last-Exam benchmarks are the same project at two registers. The question the deeds raise is not sincerity — it is independence.

Consequential actions, especially where it cost something. The defining action is the Statement on AI Risk (May 30 2023): a single line equating AI to pandemics and nuclear war, organized and hosted by CAIS, signed by Altman, Hassabis, Amodei, Hinton, and Bengio. It cost CAIS nothing and bought it the central seat in the governance debate. The harder test came in 2024, when CAIS’s 501(c)(4) Action Fund co-sponsored California’s SB 1047 — a bill that would have mandated third-party safety auditing of frontier models.

At that point Hendrycks — CAIS’s director — was also an investor in and co-founder of Gray Swan AI, an AI-auditing startup positioned to supply exactly the kind of compliance auditing the bill would require. Critics surfaced the conflict; Hendrycks responded by publicly divesting his entire equity stake in Gray Swan and continuing as an unpaid advisor, stating he was doing the work “on principle to promote the public interest.”

The divestment is the value-under-cost test passing — the director gave up the equity rather than the bill. But that the test arose at all is the structural fact: the body advocating mandatory audits was led by the founder of an audit vendor.

Leadership choices. The same director, Dan Hendrycks, simultaneously holds (at symbolic $1 salaries, no equity) the post of safety advisor to xAI and, from November 2024, advisor to Scale AI — the same Scale AI that co-produced both of CAIS’s flagship outputs, the WMDP benchmark and Humanity’s Last Exam.

On governance, Jaan Tallinn is both a CAIS funder and a board member — the same existential-risk financier who backs the broader network. And the seed money was crypto-adjacent: $6.5M from the FTX Future Fund in 2022, later subject to a bankruptcy-estate clawback probe (Bloomberg, Oct 25 2023). None of these is wrongdoing on its own. Together they describe a body whose leadership sits inside the commercial and lab ecosystem it benchmarks, regulates, and warns about — managed by $1 salaries and a divestment, but never by separation.

CONDUCT verdict: CONFLICTED — a sincere safety project whose flagship benchmarks are co-produced with a lab its own director advises, whose audit-mandate advocacy coincided with its director founding an audit vendor (resolved only by public divestment), and whose seed capital and board sit inside the existential-risk funding network it speaks for.

Sources: Center for AI Safety — About; Statement on AI Risk — CAIS; aistatement.com; Press release: Statement on AI Risk — CAIS; WMDP Benchmark — CAIS; Humanity’s Last Exam; FTX Is Probing $6.5 Million Paid to Center for AI Safety in 2022 — Bloomberg, Oct 25 2023; Center for AI Safety — Wikipedia; Center for AI Safety Action Fund; Dan Hendrycks, Elon Musk’s AI safety advisor, adds role at Scale AI — Fortune, 13 Nov 2024; Dan Hendrycks — Wikipedia; Dan Hendrycks divestment statement — X, 25 Jul 2024; Safe and Secure Innovation for Frontier Artificial Intelligence Models Act (SB 1047) — Wikipedia.