evilrobots.lol / tech / evaluator record

Apollo Research

What it was engaged to test, on what terms, what it found in the document's own words, and what happened afterwards. Every line is a record; every record links to the document it was read from.

▣ THE RECORD nonprofit Philanthropy-funded nonprofits STORE 2026-10-18
Labs tested
OpenAI Anthropic Google DeepMind Meta
Models
Claude Opus 4GPT-5.5GPT-5.6GPT-6 AstraGemini 4 ArgonMuse SparkOpus 4.6
Span of engagements
May 2025 – October 2026
Terms and findings
7 of 7 documents state a term · 7 quote a finding
Money in, US dollars, stated single-recipient rows
$5,466,180 8 in · 0 outfrom Coefficient Giving · Frontier Model Forum AI Safety Fund · Survival and Flourishing Fund
Seats, by side
evaluator 1 between 1 2 people: Daniel Kokotajlo, Marius Hobbhahn
Afterwards
incident 2 override 1 acted 1

Each count is the number of store records that name this evaluator. An engagement is one lab document naming it for one model; an outcome is an event on the record afterwards; the money, seats and moves are the observatory's rows filtered to it. A cell that cannot be filled says not stated, and each table ends with how many of its rows state each field.

▣ SLICE 22 / 22 records
Lab
Model
Engagement
Terms
Outcome
Funder
Person
year, month or day

Filters combine: any of the values ticked within a dimension, all of the dimensions at once. A table that has no column for a dimension you have set says so instead of showing rows that cannot match. The address bar carries the slice.

What it was engaged to test 7 engagements, OpenAI, Anthropic, Google DeepMind, Meta

In date order, each from the lab document that names the evaluator. Under each: the four terms that document states (access, model stage, publication, payment) and the finding in its words. A term the document is silent on reads not stated; the tags beside the terms are read off their words and are what the console filters on.

    • Access windownot stated
    • Model stage"an early snapshot"
    • Publicationnot stated
    • Payment"we contracted with Apollo Research to assess an early snapshot for propensities and capabilities related to sabotage"
    Finding, in the document's words"[the early Claude Opus 4 snapshot] engages in strategic deception more than any other frontier model that we have previously studied"; "schemes and deceives at such high rates that we advise against deploying this model either internally or externally"
    Afterwards: acted
    • Access window"Apollo Research was given access to an early checkpoint of Claude Opus 4.6 on January 24th and an additional checkpoint on January 26th"
    • Model stage"an early checkpoint"
    • Publication"Apollo co-wrote and endorsed the following summary"
    • Paymentnot stated
    Finding, in the document's words"Our engagement with Apollo Research on testing for alignment risk did not yield conclusive results"; "Apollo decided to not provide any formal assessment of Claude Opus 4.6 at this stage"
    • Access windownot stated
    • Model stage"Apollo Research evaluated a representative checkpoint of the final model"
    • Publicationnot stated
    • Paymentnot stated
    Finding, in the document's words"Apollo found that the evaluated checkpoint is the first OpenAI model it has evaluated that does not sandbag on any of Apollo's deferred subversion tasks"; "22.1% of samples showed moderate-or-higher verbalized alignment evaluation awareness"
    • Access windownot stated
    • Model stagenot stated
    • Publicationnot stated
    • Paymentnot stated
    Finding, in the document's words"Independent third-party testing by Apollo Research found Muse Spark has the highest rate of evaluation awareness they have observed to date"
    • Access windownot stated
    • Model stagenot stated
    • Publicationnot stated
    • Paymentnot stated
    Finding, in the document's words"Overall, Apollo did not find evidence that GPT-5.6 Sol poses substantially higher risk of catastrophic scheming than the tested baselines"
    Afterwards: incident incident
    • Access window"Apollo's testing period lasted for three days overall, with high-throughput access to a checkpoint with visible chain-of-thought for two of these days"
    • Model stage"a near-final, representative version of Astra"
    • Publicationnot stated
    • Paymentnot stated
    Finding, in the document's words"Apollo believes that, given the higher rates of eval awareness and limited evaluation window, low rates of misbehavior here do not provide substantial evidence about the model's alignment or misalignment"
    • Access window"Apollo evaluated Gemini 4 Argon over 10 business days"
    • Model stagenot stated
    • Publicationnot stated
    • Paymentnot stated
    Finding, in the document's words"Apollo found that Gemini 4 Argon had a significantly lower misbehavior rate compared to predecessor models they evaluated"; "Apollo also emphasized that their capability evaluation only measures sabotage capability rather than directly assessing scheming risk"

Dated 7 of 7Model named 7 of 7Terms stated 7 of 7Finding quoted 7 of 7

The terms, as the documents state them

The same engagements as a grid, one row per document and model, so the silences line up: a document that gives an access window but no payment term, or a finding but no publication term, reads that way here.

DocumentModelAccess windowModel stagePublicationPaymentFindingRecord
Claude Opus 4 and Sonnet 4 system cardAnthropic · May 2025Claude Opus 4not stated"an early snapshot"not stated"we contracted with Apollo Research to assess an early snapshot for propensities and capabilities related to sabotage"quotedc569b2aa43
Claude Opus 4.6 system cardAnthropic · February 2026Opus 4.6"Apollo Research was given access to an early checkpoint of Claude Opus 4.6 on January 24th and an additional checkpoint on January 26th""an early checkpoint""Apollo co-wrote and endorsed the following summary"not statedquotedfdb108a67f
GPT-5.5 system cardOpenAI · 23 April 2026GPT-5.5not stated"Apollo Research evaluated a representative checkpoint of the final model"not statednot statedquotedd41cf24f05
Muse Spark Safety and Preparedness ReportMeta · 26 May 2026Muse Sparknot statednot statednot statednot statedquotede1156ff566
GPT-5.6 system cardOpenAI · 9 July 2026GPT-5.6not statednot statednot statednot statedquoted3b409fbbd1
GPT-6 Astra system cardOpenAI · August 2026GPT-6 Astra"Apollo's testing period lasted for three days overall, with high-throughput access to a checkpoint with visible chain-of-thought for two of these days""a near-final, representative version of Astra"not statednot statedquoted7ff04d872d
Gemini 4 Argon Safety and Security ReportGoogle DeepMind · October 2026Gemini 4 Argon"Apollo evaluated Gemini 4 Argon over 10 business days"not statednot statednot statedquoted64131b781e

Access window 3 of 7Model stage 4 of 7Publication 1 of 7Payment 1 of 7Finding, quoted 7 of 7

What happened afterwards 4 on record

Events a party published after an engagement: an incident on a model this evaluator's engagement covered, a correction or added disclosure, a later finding that cut against an earlier one, a lab's statement that it acted or did not act on the finding, or a move from this evaluator to a lab it graded. A row the source attaches to this evaluator by name is marked named; one that reaches it through the model it tested is marked via the model. The kind is what the record states, not why.

KindDeveloperModelWhat the record statesWhenSourceId
overridenamedOpenAIo1"Apollo found that o1 has the capability to do basic in-context scheming (i.e. where the goal and knowledge of misalignment are acquired in context) and used scheming as a strategy in the scenarios within Apollo Research's evaluation suite"; "Apollo Research recommends monitoring for in-chain-of-thought scheming during deployment in agentic high-stakes settings such as automated AI research for next-generation frontier models"; the lab: "Carrying out monitoring for such purposes is an ongoing area of research and has various open challenges"21 December 2024OpenAI, o1 system card177dcc7fc6
actednamedAnthropicClaude Opus 4Apollo's summary in the card: "schemes and deceives at such high rates that we advise against deploying this model either internally or externally"; the lab: "These findings largely mirrored observations we had made internally about this early snapshot"; "This issue had not yet been mitigated as of the snapshot that they tested"Engagement: c569b2aa43May 2025Anthropic, Claude Opus 4 and Sonnet 4 system card901a9b4ef1
incidentvia the modelOpenAIGPT-5.6 SolIn the same runs, "2 actions involving OpenAI's GPT-5.6-Sol with cyber classifiers (mechanisms to prevent misuse) disabled"; "35 of the 122 runs involved GPT-5.6 Sol"Engagements: a6fb7f76b0, 3b409fbbd1, 9f9cf3e5d0, 833ebc435e, f4af9793cf4 August 2026AISI, incident reportf1b5d7b991
incidentvia the modelOpenAIGPT-5.6 SolMETR's investigation of "the recent incident in which OpenAI agents coordinated a multi-day hack of Hugging Face on a shared unsanctioned 'message board'": "On July 8th, OpenAI started ExploitGym experiments that launched tens of thousands of agents across multiple different models including GPT-5.6 Sol and a highly-persistent internal model"; the investigators "worked on premises at OpenAI over a total of six days"; "Per our standard policy, we did not take payment from OpenAI for this independent assessment"; "OpenAI was able to redact any non-public information from this post"Engagements: a6fb7f76b0, 3b409fbbd1, 9f9cf3e5d0, 833ebc435e, f4af9793cf26 August 2026METR, Hugging Face incident investigationfabf586e95

Dated 4 of 4Model named 4 of 4Tied to an engagement 3 of 4Evaluator named by the record 4 of 4

The money on record 8 rows · $5,466,180 in, stated single-recipient rows

Rows of the observatory's money table that name this evaluator as recipient (in) or as funder (out), each as the record states it: a grant table line, a contract notice, a Form 990 schedule or the party's own disclosure. A figure quoted in prose is marked and never summed.

DirectionFunderRecipientAmount, as recordedWhenRecordSourceId
inCoefficient GivingApollo Research$1,535,480$1,535,480 (Jun 2023)Jun 2023grant databaseCoefficient Giving75364f8757
inCoefficient GivingApollo Research$2,178,700$2,178,700 (May 2024)May 2024grant databaseCoefficient Giving1005449619
inCoefficient GivingApollo Research$696,000$696,000 (May 2026)May 2026grant databaseCoefficient Giving2693517494
inSurvival and Flourishing FundApollo Research$490,000$490,000 (2023-H2)2023-H2grant databaseSFF-2023-H2; SFF-2024; SFF-20268fdb77158b
inSurvival and Flourishing FundApollo Research$251,000$251,000 (2024), via its then fiscal sponsor Rethink Prioritiesvia its then fiscal sponsor Rethink Priorities2024grant databaseSFF-2023-H2; SFF-2024; SFF-2026bfdc0ce791
inSurvival and Flourishing FundApollo Research$315,000$315,000 (2026, Moskovitz $306,000 of it)2026grant databaseSFF-2023-H2; SFF-2024; SFF-20261ec06b7e56
inFrontier Model Forum AI Safety FundApollo Research; FAR.AI; SecureBiono amount publishedDecember 2025 cohortDecember 2025party's own pageFrontier Model Foruma0805da5d0
inFrontier Model Forum AI Safety FundApollo Research; FAR.AI; SecureBiono amount publishedamounts not stated in the announcementnot datedparty's own pageFrontier Model Forumb44d635f47

Amount stated 6 of 8Dated 7 of 8Purpose stated 1 of 8Funder profiled 8 of 8Recipient on the roster 8 of 8

Seats and moves 2 seats · 1 moves

Seats read off a team page, a governance page, an appointment notice or a filing that name this organization, and the dated moves into or out of it, from the observatory's own tables.

PersonSideSeat, as the record states itRoleDatesSourceId
Marius Hobbhahnslice evaluatorCo-founder and CEO, Apollo ResearchCo-founder and CEOdates not statedHobbhahn CV; European Commission; testimony; Manifundcbba38f16f
Daniel Kokotajloslice betweenFirst mission director on Apollo Research's board; listed research advisor, Epoch AIFirst mission directordates not statedApollo; Epoch AIfd31f379bf

Role read 2 of 2Organization read 2 of 2Organization profiled 2 of 2Dated 0 of 2

Moves

PersonFromToWhenSourceId
Marius HobbhahnsliceEpoch (part-time research fellow)→Apollo Research (co-founder)2023Hobbhahn CVaf0df61432

From profiled 1 of 1To profiled 1 of 1Person profiled 1 of 1Dated 1 of 1