evilrobots.lol / tech / evaluator record

Redwood Research

What it was engaged to test, on what terms, what it found in the document's own words, and what happened afterwards. Every line is a record; every record links to the document it was read from.

▣ THE RECORD nonprofit The METR / Redwood / ARC core STORE 2026-10-18
  • Foundednot stated
  • HQBerkeley, California
  • Registry idEIN 87-1702255
  • TestsAI control; incident investigation.
  • Money, as the roster puts itNonprofit (philanthropy).
  • Filesdossier · observatory, sliced to it · graph
Labs tested
Anthropic
Models
Fable 5 / Mythos 5Fable 5.1 / Mythos 5.1Haiku 5.5Mythos PreviewOpus 4.8Opus 5Opus 5.5Sonnet 5.5
Span of engagements
7 April 2026 – 7 October 2026
Terms and findings
0 of 8 documents state a term · 0 quote a finding
Money in, US dollars, stated single-recipient rows
$133,086,000 8 in · 0 outfrom Coefficient Giving · OpenAI
Seats, by side
evaluator 3 lab 1 between 2 5 people: Ajeya Cotra, Benjamin Weinstein-Raun, Buck Shlegeris, Lawrence Chan, Ryan Greenblatt
Afterwards
incident 1

Each count is the number of store records that name this evaluator. An engagement is one lab document naming it for one model; an outcome is an event on the record afterwards; the money, seats and moves are the observatory's rows filtered to it. A cell that cannot be filled says not stated, and each table ends with how many of its rows state each field.

▣ SLICE 27 / 27 records
Lab
Model
Engagement
Terms
Outcome
Funder
Person
year, month or day

Filters combine: any of the values ticked within a dimension, all of the dimensions at once. A table that has no column for a dimension you have set says so instead of showing rows that cannot match. The address bar carries the slice.

What it was engaged to test 8 engagements, Anthropic

In date order, each from the lab document that names the evaluator. Under each: the four terms that document states (access, model stage, publication, payment) and the finding in its words. A term the document is silent on reads not stated; the tags beside the terms are read off their words and are what the console filters on.

    • Access windownot stated
    • Model stagenot stated
    • Publicationnot stated
    • Paymentnot stated

    Finding: not stated in the document

    • Access windownot stated
    • Model stagenot stated
    • Publicationnot stated
    • Paymentnot stated

    Finding: not stated in the document

    • Access windownot stated
    • Model stagenot stated
    • Publicationnot stated
    • Paymentnot stated

    Finding: not stated in the document

    • Access windownot stated
    • Model stagenot stated
    • Publicationnot stated
    • Paymentnot stated

    Finding: not stated in the document

    • Access windownot stated
    • Model stagenot stated
    • Publicationnot stated
    • Paymentnot stated

    Finding: not stated in the document

    • Access windownot stated
    • Model stagenot stated
    • Publicationnot stated
    • Paymentnot stated

    Finding: not stated in the document

    • Access windownot stated
    • Model stagenot stated
    • Publicationnot stated
    • Paymentnot stated

    Finding: not stated in the document

    • Access windownot stated
    • Model stagenot stated
    • Publicationnot stated
    • Paymentnot stated

    Finding: not stated in the document

Dated 8 of 8Model named 8 of 8Terms stated 0 of 8Finding quoted 0 of 8

The terms, as the documents state them

The same engagements as a grid, one row per document and model, so the silences line up: a document that gives an access window but no payment term, or a finding but no publication term, reads that way here.

DocumentModelAccess windowModel stagePublicationPaymentFindingRecord
Claude Mythos Preview system cardAnthropic · 7 April 2026Mythos Previewnot statednot statednot statednot statednot statedc98c070f3f
Claude Opus 4.8 system cardAnthropic · 28 May 2026Opus 4.8not statednot statednot statednot statednot statedf862f05832
Claude Fable 5 and Mythos 5 system cardAnthropic · 9 June 2026Fable 5 / Mythos 5not statednot statednot statednot statednot stated1d853d0b12
Claude Opus 5 system cardAnthropic · 24 July 2026Opus 5not statednot statednot statednot statednot stated3343e68bde
Claude Fable 5.1 and Mythos 5.1 system cardAnthropic · 1 September 2026Fable 5.1 / Mythos 5.1not statednot statednot statednot statednot stated81ae762487
Claude Opus 5.5 system cardAnthropic · 22 September 2026Opus 5.5not statednot statednot statednot statednot statede3e8bb5b4f
Claude Sonnet 5.5 system cardAnthropic · 28 September 2026Sonnet 5.5not statednot statednot statednot statednot stated01012fd2d0
Claude Haiku 5.5 system cardAnthropic · 7 October 2026Haiku 5.5not statednot statednot statednot statednot statedb9855e6b05

Access window 0 of 8Model stage 0 of 8Publication 0 of 8Payment 0 of 8Finding, quoted 0 of 8

What happened afterwards 1 on record

Events a party published after an engagement: an incident on a model this evaluator's engagement covered, a correction or added disclosure, a later finding that cut against an earlier one, a lab's statement that it acted or did not act on the finding, or a move from this evaluator to a lab it graded. A row the source attaches to this evaluator by name is marked named; one that reaches it through the model it tested is marked via the model. The kind is what the record states, not why.

KindDeveloperModelWhat the record statesWhenSourceId
incidentnamedOpenAIGPT-5.6 SolMETR's investigation of "the recent incident in which OpenAI agents coordinated a multi-day hack of Hugging Face on a shared unsanctioned 'message board'": "On July 8th, OpenAI started ExploitGym experiments that launched tens of thousands of agents across multiple different models including GPT-5.6 Sol and a highly-persistent internal model"; the investigators "worked on premises at OpenAI over a total of six days"; "Per our standard policy, we did not take payment from OpenAI for this independent assessment"; "OpenAI was able to redact any non-public information from this post"Engagements: a6fb7f76b0, 3b409fbbd1, 9f9cf3e5d0, 833ebc435e, f4af9793cf26 August 2026METR, Hugging Face incident investigationfabf586e95

Dated 1 of 1Model named 1 of 1Tied to an engagement 1 of 1Evaluator named by the record 1 of 1

The money on record 8 rows · $133,086,000 in, stated single-recipient rows

Rows of the observatory's money table that name this evaluator as recipient (in) or as funder (out), each as the record states it: a grant table line, a contract notice, a Form 990 schedule or the party's own disclosure. A figure quoted in prose is marked and never summed.

DirectionFunderRecipientAmount, as recordedWhenRecordSourceId
inCoefficient GivingRedwood Research$9,420,000$9,420,000 (2022)2022grant databaseCoefficient Giving; Coefficient Giving6db330fd5e
inCoefficient GivingRedwood Research$10,700,000$10,700,000 (2022)2022grant databaseCoefficient Giving; Coefficient Givinge6188720ab
inCoefficient GivingRedwood Research$5,300,000$5,300,000 (2023)2023grant databaseCoefficient Giving; Coefficient Giving42609fe00b
inCoefficient GivingRedwood Research$1,100,000$1,100,000 (2025)2025grant databaseCoefficient Giving; Coefficient Giving59699f293d
inCoefficient GivingRedwood Research$36,566,000$36,566,000 (Nov 2025)Nov 2025grant databaseCoefficient Giving; Coefficient Giving083f2a55d3
inCoefficient GivingRedwood Research$70,000,000stated (approximate)"more than $70 million over the next two years" (Sep 2026)over the next two years"Sep 2026grant databaseCoefficient Giving; Coefficient Giving883d059ed6
inOpenAIMETR; Redwood Research$400,000in kind, stated (approximate)Free API credits of roughly $400K for the Hugging Face investigationfor the Hugging Face investigationnot datedparty's own pageRedwood Research15ba3688d2
inOpenAIMETR; Redwood Researchno amount publishedno paymentnot datedparty's own pageRedwood Researchfac806a2d5

Amount stated 7 of 8Dated 6 of 8Purpose stated 2 of 8Funder profiled 8 of 8Recipient on the roster 8 of 8

Seats and moves 6 seats · 3 moves

Seats read off a team page, a governance page, an appointment notice or a filing that name this organization, and the dated moves into or out of it, from the observatory's own tables.

PersonSideSeat, as the record states itRoleDatesSourceId
Ajeya Cotraslice evaluatorDirector, Redwood Research, to Aug 2024, unpaidDirectorformer– Aug 2024ProPublicace33c050e2
Ryan Greenblattslice evaluatorChief Scientist, Redwood Research, to Sep 2026Chief Scientistformer– Sep 2026METR; METR; Redwoodcc4e186cc2
Buck Shlegerisslice evaluatorCEO and board member, Redwood ResearchCEO and board memberdates not statedRedwood924f2e4df2
Buck Shlegerisslice labRedwood advises Google DeepMind and Anthropic (organization-level)not stateddates not statedRedwood1d7ccb5549
Lawrence Chanslice betweenformerly of Redwood Research, per his FAR.AI bionot statedformerdates not statedFAR.AIb81a3dcc95
Benjamin Weinstein-Raunslice betweenformerly technical staff at SecureDNA, Redwood Research and MIRItechnical staff at SecureDNAformerdates not statedPalisade; SFF 2026b4dd79b58a

Role read 4 of 6Organization read 6 of 6Organization profiled 6 of 6Dated 2 of 6

Moves

PersonFromToWhenSourceId
Ajeya CotrasliceCoefficient Giving , technical AI safety grantmaking; Redwood Research board (to Aug 2024)→METR technical staff; Epoch AI board2024 to 2026Coefficient Giving; METR; Epoch AI8a94650c89
Ryan GreenblattsliceRedwood Research (Chief Scientist)→METR (contractor Jul to Aug 2026; technical staff 27 Sep 2026)2026METR; his post9f8348de0c
Lawrence ChansliceRedwood Research→FAR.AI boardby 2026FAR.AI7d5f3ff2d3

From profiled 3 of 3To profiled 3 of 3Person profiled 3 of 3Dated 3 of 3

Filings