AI Safety Theater: Safety as the Capture Surface

The firms selling the AI risk are the firms selling the cure. The capture mechanism is the safety brand itself — licensing thresholds as moats, compute governance as a chokepoint, and the 'regulate me' dynamic that formalizes the incumbents' place at the head of the line.

2026-07-20 9 min read Research file
Contents

A research position. Sourced, present-all-sides. Named people and organizations carry primary citations; characterizations are attributed to whoever made them, never adopted as this page’s own voice. This is the capture-analysis companion to The AI Governance Ratchet and the public research base for The Ratchet Ch. 20 (“The Blueprint”). The named-operator conduct record — the grift-and-shaming angle — lives off-site at troll.fan; this page keeps the structural analysis.

The load-bearing observation is not that AI safety is fake. Some of the risk is real, some of the research is serious, and the question of what a frontier model should refuse is a genuine one. The observation is narrower and harder to dislodge: the same firms that market the risk are the firms that market the cure, and the cure they propose — licensing, compute thresholds, conformity regimes — is, by construction, a barrier to entry they are best positioned to clear. The safety brand is not a side effect of the business. In the capture frame, it is the capture surface. This page scores that structure, not the private motives of the people inside it.

The “regulate me” dynamic

The tell is that the request for regulation comes from the regulated, unprompted.

On 16 May 2023, OpenAI CEO Sam Altman testified before the Senate Judiciary Subcommittee on Privacy, Technology, and the Law and proposed a federal licensing regime for frontier AI: government permission required before training models above a capability threshold, a new agency to issue the licenses, safety standards applicants must meet (Senate Judiciary hearing record). He had not been subpoenaed. Senator Blumenthal remarked on how unusual it was for an industry to ask to be regulated. The structural effect of a licensing regime, whatever the intent behind it, is to limit frontier development to organizations that can navigate a licensing process — well-funded incumbents, not graduate students or open-source collectives. Critics characterize this as a moat; the moat reading describes the effect, not an imputed motive.

By late 2025 the dynamic had become explicit on camera. Anthropic CEO Dario Amodei — running a lab that markets itself on safety while competing at the frontier — sat for CBS 60 Minutes and was asked by Anderson Cooper who elected him and Sam Altman to make these decisions. His answer: “No one. No one. Honestly, no one.” He said he was “deeply uncomfortable with these decisions being made by a few companies, by a few people,” and offered that discomfort as the reason he had “always advocated for responsible and thoughtful regulation of the technology” (CBS 60 Minutes transcript, aired 16 Nov 2025; Fortune summary). The exchange is the ratchet compressed to a single beat: the unelected few who hold the levers concede that no one should, name themselves as the problem, and propose as the remedy a licensing regime that formalizes their place at the head of the line. Whether that is candor or positioning is exactly what the record cannot settle — and for the capture analysis, it does not need to. The structure produces the same outcome either way.

Safety regulation as a moat: the threshold is the barrier

The clearest instance of safety-as-moat is a number. The EU AI Act’s general-purpose-AI regime attaches “systemic risk” duties to models trained above a presumption of 10^25 floating-point operations; open-source models get partial exemptions — but not the frontier ones above the threshold (EUR-Lex, Regulation (EU) 2024/1689; EU AI Act explorer). Below the line, you may open-source freely. Above it, you need the compliance apparatus of a major corporation. The threshold does not regulate a harm; it regulates a scale. It makes it economically irrational to open-source a frontier model into the European market — which is to say, it draws the moat exactly where the incumbents already sit.

This is the general form the safety pitch takes when it becomes policy. The stated object is a catastrophic capability. The operative object is a capability scale that only a handful of firms can reach — so the rule that governs the scale governs the firms below it out of the market, and leaves the firms at the frontier holding a license the newcomer cannot afford. The precedent is Facebook’s 2018 call for internet regulation after Cambridge Analytica, widely read by critics across the spectrum as an attempt to lock in a market position: regulation an incumbent can afford and a competitor cannot is not a burden, it is a barrier (the parallel is developed in Ch. 20 and the governance dossier).

The compute-governance licensing ratchet

The hardware layer is where the licensing logic is most advanced, because compute is a physical chokepoint you can actually meter. US chip export controls already decide who can build frontier AI at the hardware level: the October 2022 BIS rule restricted advanced logic chips and chipmaking equipment to China (CSIS), and the chokepoints are natural — TSMC fabricates roughly 90% of the most advanced chips, ASML is the sole maker of EUV lithography (ETO/CSET). The specific rules churn — the January 2025 AI Diffusion Rule that would have globalized the tiering was rescinded by BIS on 13 May 2025 before it took effect (BIS), and Nvidia’s H20 was allowed back into China in July 2025 under a revenue-share arrangement (CNBC). The targeting keeps getting adjusted. The licensing machinery and the chokepoints stay.

The frontier of the hardware layer is on-chip governance: proposals for chips that cryptographically attest to the workloads they run, firmware compute caps, tamper-resistant usage logs, and remote attestation, modeled explicitly on nuclear arms-control verification (Heim et al., arXiv). Sold as a way to verify safety compliance without seizing models, it is also — critics note — a per-GPU throttle and audit hook that a governing body could point at any workload it chose. Same infrastructure, both purposes; it does not have to choose between them.

The counter-evidence the safety-as-compute-moat argument has to survive: in January 2026’s long shadow, DeepSeek’s R1 (released January 2025, base model reportedly trained for ~$5.6M on export-restricted H800s) challenged the compute-moat premise directly, and Nvidia shed ~$589B in market cap on 27 January 2025 (CNBC). If a restricted lab can reach the frontier cheaply and open-source it, the claim that only the richest firms can build safely — the claim the licensing regime rests on — is the part that cracks. The chokepoints are real; the durable-capability-gap story built on top of them is contestable.

The documented lobbying spend

The capture argument does not require a hidden conspiracy; it requires a disclosure form. The firms that can afford compliance lobby for the regulations that make compliance mandatory, and the spend is public.

The point is not that this is corruption. It is that it is the system functioning as designed: companies participate in the regulatory process, advocate their interests, and the result is regulation the largest companies helped write and the smallest cannot afford to implement. That the same machinery built for safety is, by construction, a content-monitoring and market-gating capability is a structural observation, not an allegation.

Alignment-as-marketing and red-team-as-assurance

The corporate-practice layer completes the loop. The safety-branded lab publishes a scaling policy — Anthropic’s Responsible Scaling Policy defines AI Safety Levels (ASL) and commitments that gate capability behind internal safety evaluations (Anthropic RSP) — and OpenAI runs an analogous Preparedness Framework. As voluntary self-governance these are real internal constraints; as public artifacts they are also assurance products. The red-team report, the model card, the safety evaluation, the “we tested it and released it anyway” disclosure: each is a genuine engineering practice and a marketing surface that says “we are the responsible ones,” which is precisely the differentiator a licensing regime would convert into a legal moat. The capture reading is not that the safety work is insincere. It is that the safety work doubles as the brand, and the brand doubles as the barrier — and a standard, unlike a statute, has no sunset clause and no electorate (see the standards analysis in the governance dossier).

The defense (at full strength)

The strongest “this is proportionate governance, not capture” case, given equal weight:

  1. Some risk is real and the labs closest to it know most about it. Wanting it governed is not evidence of bad faith; it may be the responsible position.
  2. The mandates did roll back. Trump’s EO 14179 revoked Biden’s EO 14110; SB 1047 was vetoed; the AI Diffusion Rule was rescinded. A one-way ratchet would not reverse.
  3. Voluntary is voluntary. NIST AI RMF and the GPAI Code of Practice bind no one by force.
  4. DeepSeek undercuts the moat story — which is awkward for the capture thesis’s cynical read too: if the moat is that easy to bypass, it is a bad moat.
  5. Present harms are real. The AI-ethics camp’s point — bias, surveillance, labor, concentration — argues for more accountability, not less, and not all of it favors incumbents.

The counter kept for balance: “voluntary” standards become de-facto mandatory through procurement and insurance; revoked executive orders leave the scaffolding (NIST, the AI Safety Institute, export controls) intact; and the institutional infrastructure persists because the EU AI Act requires it regardless of what any single administration does. Both cases are on the table; the documented record does the work.

Where it converges

Across the licensing threshold, the compute chokepoint, the disclosed lobbying, and the safety brand, the apparatus lands on one point. The genuinely contested question is not whether a control layer gets built, or even whether safety is real. It is who operates the layer, and whose brand becomes the license. The firm that sells the risk and the cure does not need the risk to be fake for the arrangement to function as capture. It only needs the cure to be shaped like a moat — and the threshold, the chokepoint, and the standard are all shaped exactly like one.

researchai-governancethe-ratchetregulatory-capturecomputeai-safety
← OlderSecurity Industry Capture & Surveillance Vendors — Research Reference

Get updates on the Evil Robots series

Newsletter essays on AI escape, deception, and the humans who built them.