GRAY SWAN

Institutional Archetype

THE TESTER WHOSE CO-FOUNDER SITS ON A LAB’S BOARD — Gray Swan runs a large adversarial red-teaming network, sells “the benchmarks frontier labs run before they ship,” and sells enterprise products that filter what AI systems say and do. One of its co-founders, its chief scientist, sits on OpenAI’s board. Any one of these is ordinary. The file is open because all three sit in one company: the evaluator, the vendor of the fix, and a seat at the table of a lab it tests.

Mandate & Origin

“Gray Swan is on a mission to empower the world to use AI safely and securely. Founded by world-leading AI safety and security researchers from Carnegie Mellon”; co-founder and chief executive Matt Fredrikson, co-founder and chief scientist Zico Kolter (Gray Swan).

Funding & Backers

A private company selling to enterprises and model builders (Gray Swan); its investors are not itemized in the sources above and this file asserts none.

Position in the Apparatus

• For the labs. “Arena: The world’s largest adversarial AI red teaming network”; “Adversarial Evaluation: The benchmarks frontier labs run before they ship”; tests “against attacks discovered by 15,000+ adversarial researchers” (Gray Swan).
• For everyone else. “Cygnal: Block adversarial inputs and unsafe outputs in real time”; “Shade: Break your AI before attackers do” (Gray Swan).
• The board seat. OpenAI announced on August 8, 2024 that Kolter, head of Carnegie Mellon’s Machine Learning Department, had joined its board, noting that he is “currently … Chief Technical Advisor at Gray Swan, a startup specializing in AI safety and security” (OpenAI).

Actions & Leadership Choices

CONDUCT: CONFLICTED — EVALUATOR, VENDOR, AND LAB-BOARD CONNECTION. No wrongdoing is alleged, and OpenAI disclosed the Gray Swan role when it announced the appointment. The structural point is the combination: a company that finds the weaknesses, sells the filter for them, and has a co-founder on the board of one of the labs whose models it tests. Whether its work for that lab is arm’s length is a question its disclosures would have to answer; the sources above do not.

Reach Assessment

Institutional: Pre-ship adversarial testing for frontier labs, by its own account, and a direct line to one lab’s board. Memetic: Its public red-teaming competitions put jailbreak results in front of a large audience.

Sources: About — Gray Swan; Gray Swan — home; Zico Kolter Joins OpenAI’s Board of Directors — OpenAI (Aug 8, 2024).