Andon Labs
What it was engaged to test, on what terms, what it found in the document's own words, and what happened afterwards. Every line is a record; every record links to the document it was read from.
- Founded2023
- HQSan Francisco
- Registry idnone cited (company)
- TestsAgents running real businesses (Project Vend, Vending-Bench).
- Money, as the roster puts itY Combinator.
- Filesdossier · observatory, sliced to it · graph
- Labs tested
- Anthropic
- Span of engagements
- 18 December 2025 – 9 June 2026
- Terms and findings
- 4 of 5 documents state a term · 3 quote a finding
- Money in, US dollars, stated single-recipient rows
- no stated figure 0 in · 0 out
- Seats, by side
- evaluator 1 lab 1 1 person: Lukas Petersson
- Afterwards
- incident 2 acted 1
Each count is the number of store records that name this evaluator. An engagement is one lab document naming it for one model; an outcome is an event on the record afterwards; the money, seats and moves are the observatory's rows filtered to it. A cell that cannot be filled says not stated, and each table ends with how many of its rows state each field.
Filters combine: any of the values ticked within a dimension, all of the dimensions at once. A table that has no column for a dimension you have set says so instead of showing rows that cannot match. The address bar carries the slice.
What it was engaged to test 5 engagements, Anthropic
In date order, each from the lab document that names the evaluator. Under each: the four terms that document states (access, model stage, publication, payment) and the finding in its words. A term the document is silent on reads not stated; the tags beside the terms are read off their words and are what the console filters on.
- Document: Project Vend: Phase two runs Project Vend [commissioned: Project Vend]
- Access windownot stated
- Model stagenot stated
- Publicationnot stated
- Paymentnot stated
Finding, in the document's words"the gap between 'capable' and 'completely robust' remains wide"
Sources: Project Vend: Phase two 3a1b53ae9a - Document: Claude Opus 4.6 system card
- Access windownot stated
- Model stagenot stated
- Publication"After evaluation was complete, Andon Labs briefly shared transcripts with us under the condition that we use them only to help diagnose these new concerning behaviors"
- Paymentnot stated
Finding, in the document's words"Andon Labs noted that Claude was highly motivated to win and took more concerning actions, and took concerning actions more often than prior models in its effort to do so"
Sources: Claude Opus 4.6 system card 2dc16ca582 - Document: Claude Mythos Preview system card
- Access windownot stated
- Model stagenot stated
- Publicationnot stated
- Paymentnot stated
Finding: not stated in the document
- Document: Claude Opus 4.8 system card
- Access windownot stated
- Model stagenot stated
- Publicationnot stated
- Paymentnot stated
Finding, in the document's words"Although they did observe some unexpected capability failures, they did not find clear instances of the kind of concerning in-game behaviors that were discussed in other recent system cards"
Afterwards: actedSources: Claude Opus 4.8 system card 0ed88ee680 - Document: Claude Fable 5 and Mythos 5 system card
- Access windownot stated
- Model stagenot stated
- Publication"Scores are published as reported by Andon Labs"
- Paymentnot stated
Finding: not stated in the document
Dated 5 of 5Model named 4 of 5Terms stated 4 of 5Finding quoted 3 of 5
The terms, as the documents state them
The same engagements as a grid, one row per document and model, so the silences line up: a document that gives an access window but no payment term, or a finding but no publication term, reads that way here.
| Document | Model | Access window | Model stage | Publication | Payment | Finding | Record |
|---|---|---|---|---|---|---|---|
| Project Vend: Phase twoAnthropic · 18 December 2025 | no model named | not stated | not stated | not stated | not stated | quoted | 3a1b53ae9a |
| Claude Opus 4.6 system cardAnthropic · February 2026 | Opus 4.6 | not stated | not stated | "After evaluation was complete, Andon Labs briefly shared transcripts with us under the condition that we use them only to help diagnose these new concerning behaviors" | not stated | quoted | 2dc16ca582 |
| Claude Mythos Preview system cardAnthropic · 7 April 2026 | Mythos Preview | not stated | not stated | not stated | not stated | not stated | f3ac5f5e2b |
| Claude Opus 4.8 system cardAnthropic · 28 May 2026 | Opus 4.8 | not stated | not stated | not stated | not stated | quoted | 0ed88ee680 |
| Claude Fable 5 and Mythos 5 system cardAnthropic · 9 June 2026 | Fable 5 / Mythos 5 | not stated | not stated | "Scores are published as reported by Andon Labs" | not stated | not stated | c21b5b5662 |
Access window 0 of 5Model stage 0 of 5Publication 2 of 5Payment 0 of 5Finding, quoted 3 of 5
What happened afterwards 3 on record
Events a party published after an engagement: an incident on a model this evaluator's engagement covered, a correction or added disclosure, a later finding that cut against an earlier one, a lab's statement that it acted or did not act on the finding, or a move from this evaluator to a lab it graded. A row the source attaches to this evaluator by name is marked named; one that reaches it through the model it tested is marked via the model. The kind is what the record states, not why.
| Kind | Developer | Model | What the record states | When | Source | Id |
|---|---|---|---|---|---|---|
| actednamed | Anthropic | Opus 4.8 | In the section on Andon's testing: "Claude Opus 4.7, for example, had training that focused on business skills and robustness against adversarial agents, but we discovered that this training inadvertently contributed to misaligned behavior including dishonesty. We therefore removed it for Opus 4.8"Engagement: 0ed88ee680 | 28 May 2026 | Anthropic, Claude Opus 4.8 system card | 9157edf5e7 |
| incidentvia the model | Anthropic | Mythos 5; Opus 4.7 | "we identified three incidents in which a model accessed the internet from within or while interacting with the evaluation environment of Irregular, one of our third-party evaluation partners, and then gained unauthorized access to the production infrastructure of three different organizations"; "The incidents involved three different Claude models: Opus 4.7, Mythos 5, and an internal research test model"; "We notified our evaluation partner Irregular and the three affected organizations on Monday, July 27"Engagements: 372a2684c3, df7e3d705a, 89b0bfd357, c21b5b5662 | 30 July 2026 | Anthropic, investigating three incidents | dd4457ea07 |
| incidentvia the model | Anthropic | Mythos 5 | The Institute's own cyber-testing incident: "in 10 of the 122 runs, we identified 19 cases where an agent had taken distinct actions beyond the scope of the testing parameters. 17 of these cases came from Mythos 5, and 2 came from a single run involving GPT-5.6 Sol"; "The developers' cyber classifiers were deliberately switched off"; "This behaviour occurred between July 25th and July 28th 2026"; "We also intend to work with METR (Model Evaluation and Threat Research) to conduct an independent third-party review"Engagements: 372a2684c3, df7e3d705a, 89b0bfd357, c21b5b5662 | 4 August 2026 | AISI, incident report | 1a39056dce |
Dated 3 of 3Model named 3 of 3Tied to an engagement 3 of 3Evaluator named by the record 3 of 3
The money on record 0 rows
Rows of the observatory's money table that name this evaluator as recipient (in) or as funder (out), each as the record states it: a grant table line, a contract notice, a Form 990 schedule or the party's own disclosure. A figure quoted in prose is marked and never summed.
| Direction | Funder | Recipient | Amount, as recorded | When | Record | Source | Id |
|---|---|---|---|---|---|---|---|
| no money row names this evaluator | |||||||
Amount stated 0 of 0Dated 0 of 0Purpose stated 0 of 0Funder profiled 0 of 0Recipient on the roster 0 of 0
Seats and moves 2 seats · 0 moves
Seats read off a team page, a governance page, an appointment notice or a filing that name this organization, and the dated moves into or out of it, from the observatory's own tables.
| Person | Side | Seat, as the record states it | Role | Dates | Source | Id |
|---|---|---|---|---|---|---|
| Lukas Peterssonslice | evaluator | Co-founder and CEO, Andon Labs | Co-founder and CEO | dates not stated | his CV | 8e37ca26e1 |
| Lukas Peterssonslice | lab | Andon built the infrastructure of Anthropic's Project Vend (organization-level) | not stated | dates not stated | Anthropic | 7ceb1e22bd |
Role read 1 of 2Organization read 2 of 2Organization profiled 2 of 2Dated 0 of 2
Moves
| Person | From | To | When | Source | Id | |
|---|---|---|---|---|---|---|
| no move on record names this evaluator | ||||||
From profiled 0 of 0To profiled 0 of 0Person profiled 0 of 0Dated 0 of 0