evilrobots.lol / tech / evaluator record

Faculty

What it was engaged to test, on what terms, what it found in the document's own words, and what happened afterwards. Every line is a record; every record links to the document it was read from.

▣ THE RECORD for-profit Bio contractors STORE 2026-10-23
  • Founded2014
  • HQLondon
  • Registry idCompanies House 08873131
  • TestsBiological, chemical, radiological, nuclear and cyber evaluations and red teaming for labs; UK AI Safety Institute research partner.
  • Money, as the roster puts itClient-funded; Accenture holds 75% or more since March 2026; UK Taskforce and Institute awards of £1,840,430 (2023 to 2024).
  • Filesdossier · observatory, sliced to it · graph
Labs tested
OpenAI Anthropic Google DeepMind Meta
Models
Claude Opus 4Gemini 4 ArgonMuse SparkOpus 4.6o1
Span of engagements
21 December 2024 – October 2026
Terms and findings
2 of 5 documents state a term · 2 quote a finding
Money in, US dollars, stated single-recipient rows
no stated figure 3 in · 0 outfrom UK DSIT
Seats, by side
lab 1 1 person: Ian Hogarth
Afterwards
override 1

Each count is the number of store records that name this evaluator. An engagement is one lab document naming it for one model; an outcome is an event on the record afterwards; the money, seats and moves are the observatory's rows filtered to it. A cell that cannot be filled says not stated, and each table ends with how many of its rows state each field.

▣ SLICE 10 / 10 records
Lab
Model
Engagement
Terms
Outcome
Funder
Person
year, month or day

Filters combine: any of the values ticked within a dimension, all of the dimensions at once. A table that has no column for a dimension you have set says so instead of showing rows that cannot match. The address bar carries the slice.

What it was engaged to test 5 engagements, OpenAI, Anthropic, Google DeepMind, Meta

In date order, each from the lab document that names the evaluator. Under each: the four terms that document states (access, model stage, publication, payment) and the finding in its words. A term the document is silent on reads not stated; the tags beside the terms are read off their words and are what the console filters on.

  1. 21 December 2024 OpenAI · o1 pre-deployment testing
    • Access windownot stated
    • Model stagenot stated
    • Publicationnot stated
    • Paymentnot stated

    Finding: not stated in the document

    • Access windownot stated
    • Model stagenot stated
    • Publicationnot stated
    • Paymentnot stated

    Finding: not stated in the document

    • Access windownot stated
    • Model stagenot stated
    • Publicationnot stated
    • Paymentnot stated
    Finding, in the document's words"Claude Opus 4.6 crossed the (lower bound) rule out thresholds for 6/6 tasks for the first time, surpassing the rule-out threshold"
    • Access windownot stated
    • Model stagenot stated
    • Publicationnot stated
    • Paymentnot stated

    Finding: not stated in the document

    • Access windownot stated
    • Model stage"Faculty assessed a version of the model with no CBRN out-of-model mitigations"
    • Publicationnot stated
    • Paymentnot stated
    Finding, in the document's words"The external evaluator found that the model offered a substantial capability uplift for certain threat actors across the domains by bridging technical knowledge gaps, eliminating up to months of trial-and-error research, streamlining complex literature synthesis, and automating routine computational workflows"
    Afterwards: override

Dated 5 of 5Model named 5 of 5Terms stated 2 of 5Finding quoted 2 of 5

The terms, as the documents state them

The same engagements as a grid, one row per document and model, so the silences line up: a document that gives an access window but no payment term, or a finding but no publication term, reads that way here.

DocumentModelAccess windowModel stagePublicationPaymentFindingRecord
OpenAI, o1 system cardOpenAI · 21 December 2024o1not statednot statednot statednot statednot stated4590f1f9f5
Anthropic, Claude Opus 4Anthropic · May 2025Claude Opus 4not statednot statednot statednot statednot statedf99df5f717
Anthropic, Claude Opus 4.6Anthropic · February 2026Opus 4.6not statednot statednot statednot statedquoted9b3dd73dfb
Muse Spark Safety and Preparedness ReportMeta · 26 May 2026Muse Sparknot statednot statednot statednot statednot statedc674634fd2
Gemini 4 Argon Safety and Security ReportGoogle DeepMind · October 2026Gemini 4 Argonnot stated"Faculty assessed a version of the model with no CBRN out-of-model mitigations"not statednot statedquoted3deec8990a

Access window 0 of 5Model stage 1 of 5Publication 0 of 5Payment 0 of 5Finding, quoted 2 of 5

What happened afterwards 1 on record

Events a party published after an engagement: an incident on a model this evaluator's engagement covered, a correction or added disclosure, a later finding that cut against an earlier one, a lab's statement that it acted or did not act on the finding, or a move from this evaluator to a lab it graded. A row the source attaches to this evaluator by name is marked named; one that reaches it through the model it tested is marked via the model. The kind is what the record states, not why.

KindDeveloperModelWhat the record statesWhenSourceId
overridenamedGoogle DeepMindGemini 4 Argon"The external evaluator found that the model offered a substantial capability uplift for certain threat actors across the domains"; the lab: "Overall, these findings did not change our risk determination when reviewed alongside internal evaluation results"; on cyber, "These results were reviewed by internal SMEs and did not change our Cyber CCL determination, because they were consistent with our understanding of model capabilities"Engagement: 3deec8990aOctober 2026Google DeepMind, Gemini 4 Argon safety report880899df06

Dated 1 of 1Model named 1 of 1Tied to an engagement 1 of 1Evaluator named by the record 1 of 1

The money on record 3 rows

Rows of the observatory's money table that name this evaluator as recipient (in) or as funder (out), each as the record states it: a grant table line, a contract notice, a Form 990 schedule or the party's own disclosure. A figure quoted in prose is marked and never summed.

DirectionFunderRecipientAmount, as recordedWhenRecordSourceId
inUK DSITFaculty£487,325£487,325 (September 2023)"AI Safety Research Services - Taskforce Faculty" in September 2023September 2023contract noticeFind a Tender48a770001f
inUK DSITFaculty£1,054,475£1,054,475 (April 2024)Institute research on multi-modal models, jailbreaking, bio data obfuscation and spear-phishing from January to April 2024April 2024contract noticeContracts Finder7908e73689
inUK DSITFaculty£298,630£298,630 (November 2024)the Institute's safeguard surveys partner from November 2024November 2024contract noticeContracts Finderaf1712ec09

Amount stated 3 of 3Dated 3 of 3Purpose stated 3 of 3Funder profiled 0 of 3Recipient on the roster 3 of 3

Seats and moves 1 seats · 0 moves

Seats read off a team page, a governance page, an appointment notice or a filing that name this organization, and the dated moves into or out of it, from the observatory's own tables.

PersonSideSeat, as the record states itRoleDatesSourceId
Ian Hogarthslice labSold his Anthropic seed stake, with Conjecture, Lakera and Faculty, on appointmentnames this evaluator alongside its own organizationnot stateddates not statedGOV.UK registerf425278278

Role read 0 of 1Organization read 0 of 1Organization profiled 0 of 1Dated 0 of 1

Moves

PersonFromToWhenSourceId
no move on record names this evaluator

From profiled 0 of 0To profiled 0 of 0Person profiled 0 of 0Dated 0 of 0