SECUREBIO

Institutional Archetype

THE TEST THE MODEL PASSED — SecureBio writes the biology tests that tell the labs, and the public, whether a model can help someone make a virus. Its best-known test was built to be hard for experts: practising virologists score about one in five on questions in their own specialty. A frontier model scored about twice that. The file is open on the position. A small pandemic-security nonprofit’s benchmark has become one of the public measures of whether a frontier model knows enough virology to be dangerous.

Mandate & Origin

“Securing the future against catastrophic pandemics.” On AI: “SecureBio is developing evaluation tools to assess the potential of AI systems to exacerbate these risks, as well as exploring mitigation strategies to address them. A member of the NIST US AI Safety Consortium, SecureBio partners with model developers, non-profit groups, and policymakers to improve oversight of frontier AI models” (SecureBio). Its broader work includes pathogen detection (the former Nucleic Acid Observatory, now “SecureBio Detection”) and DNA-synthesis screening (SecureDNA).

Funding & Backers

A nonprofit taking donations (SecureBio); its funders are not itemized in the sources above and this file asserts none. See The EA to AI-Safety Funding Pipeline.

Position in the Apparatus

• The Virology Capabilities Test (April 2025). “VCT consists of 322 multimodal questions covering fundamental, tacit, and visual knowledge that is essential for practical work in virology laboratories. VCT is difficult: expert virologists with access to the internet score an average of 22.1% on questions specifically in their sub-areas of expertise. However, the most performant LLM, OpenAI’s o3, reaches 43.8% accuracy, outperforming 94% of expert virologists even within their sub-areas of specialization” (arXiv 2504.16137). Co-authors include Dan Hendrycks of the Center for AI Safety.
• The arc this test sits on, from RAND’s 2024 “no significant uplift” to the 2026 system cards’ saturated early-warning tests, is in AI and Bioweapons Uplift.

Actions & Leadership Choices

CONDUCT: PRIVATE BIORISK GRADER. No wrongdoing is alleged. The structural point is that a knowledge benchmark built by a small nonprofit stands in, in public discussion, for a question it does not directly measure: whether a model helps someone build a weapon. Epoch AI’s critique of lab biorisk evaluations makes that argument (AI and Bioweapons Uplift); SecureBio’s own abstract measures troubleshooting knowledge, and says so.

Reach Assessment

Institutional: Its benchmark is a public yardstick for frontier models’ virology knowledge, and it holds a seat in NIST’s consortium. Memetic: “The model beats 94% of virologists” is the line that travelled.

Sources: SecureBio — home; Virology Capabilities Test (VCT): A Multimodal Virology Q&A Benchmark — arXiv 2504.16137.