Princeton Holistic Agent Leaderboard
Princeton Holistic Agent Leaderboard (HAL). What it was engaged to test, on what terms, what it found in the document's own words, and what happened afterwards. Every line is a record; every record links to the document it was read from.
- Foundednot stated
- HQPrinceton
- Registry idnone cited
- TestsAgent benchmarking on a common harness with cost tracking (nine benchmarks, 26,597 rollouts), published traces, and a Reliability Dashboard; has paused adding new models; Forum member.
- Money, as the roster puts it"Coefficient Giving, Schmidt Sciences, the Princeton AI Lab, the Princeton Language and Intelligence Initiative, and the Princeton Catalysis Initiative"; "OpenAI and Google" supply API credits to evaluate their models; no amount published.
- Filesdossier · observatory, sliced to it
- Labs tested
- no lab document names it
- Models
- no model named
- Span of engagements
- none dated
- Terms and findings
- 0 of 0 documents state a term · 0 quote a finding
- Money in, US dollars, stated single-recipient rows
- $1,908,763 8 in · 0 outfrom Coefficient Giving · Google · OpenAI · Schmidt Sciences
- Seats, by side
- no seat names it
- Afterwards
- no later event on record
Each count is the number of store records that name this evaluator. An engagement is one lab document naming it for one model; an outcome is an event on the record afterwards; the money, seats and moves are the observatory's rows filtered to it. A cell that cannot be filled says not stated, and each table ends with how many of its rows state each field.
Filters combine: any of the values ticked within a dimension, all of the dimensions at once. A table that has no column for a dimension you have set says so instead of showing rows that cannot match. The address bar carries the slice.
What it was engaged to test 0 engagements
In date order, each from the lab document that names the evaluator. Under each: the four terms that document states (access, model stage, publication, payment) and the finding in its words. A term the document is silent on reads not stated; the tags beside the terms are read off their words and are what the console filters on.
No lab document on record names this evaluator for a model.
Dated 0 of 0Model named 0 of 0Terms stated 0 of 0Finding quoted 0 of 0
The terms, as the documents state them
The same engagements as a grid, one row per document and model, so the silences line up: a document that gives an access window but no payment term, or a finding but no publication term, reads that way here.
| Document | Model | Access window | Model stage | Publication | Payment | Finding | Record |
|---|---|---|---|---|---|---|---|
| no engagement on record | |||||||
Access window 0 of 0Model stage 0 of 0Publication 0 of 0Payment 0 of 0Finding, quoted 0 of 0
What happened afterwards 0 on record
Events a party published after an engagement: an incident on a model this evaluator's engagement covered, a correction or added disclosure, a later finding that cut against an earlier one, a lab's statement that it acted or did not act on the finding, or a move from this evaluator to a lab it graded. A row the source attaches to this evaluator by name is marked named; one that reaches it through the model it tested is marked via the model. The kind is what the record states, not why.
| Kind | Developer | Model | What the record states | When | Source | Id |
|---|---|---|---|---|---|---|
| no later event on record | ||||||
Dated 0 of 0Model named 0 of 0Tied to an engagement 0 of 0Evaluator named by the record 0 of 0
The money on record 8 rows · $1,908,763 in, stated single-recipient rows
Rows of the observatory's money table that name this evaluator as recipient (in) or as funder (out), each as the record states it: a grant table line, a contract notice, a Form 990 schedule or the party's own disclosure. A figure quoted in prose is marked and never summed.
| Direction | Funder | Recipient | Amount, as recorded | When | Record | Source | Id |
|---|---|---|---|---|---|---|---|
| in | Coefficient Giving | Princeton Holistic Agent Leaderboard | $863,143$863,143 (Nov 2024) to Princeton for an AI R&D benchmark led by Arvind Narayanan, via the agent-benchmarks request for proposalsto Princeton for an AI R&D benchmark led by Arvind Narayanan, via the agent-benchmarks request for proposals | Nov 2024 | grant database | Coefficient Giving; HAL | b8151eea2e |
| in | Coefficient Giving | Princeton Holistic Agent Leaderboard | $1,045,620$1,045,620 (Jul 2024) to Princeton for Karthik Narasimhan's software-engineering benchmarkto Princeton for Karthik Narasimhan's software-engineering benchmark | Jul 2024 | grant database | Coefficient Giving; HAL | 356ee01ead |
| in | Coefficient Giving | Princeton Holistic Agent Leaderboard | no amount publishedneither listing names HAL, whose own page names Coefficient Giving as a funder | not dated | grant database | Coefficient Giving; HAL | c45920c4eb |
| in | Schmidt Sciences | METR; Princeton Holistic Agent Leaderboard | no amount publishedHAL: named funder on its own page | not dated | party's own page | HAL; Schmidt Sciences | bb7cf8ceb4 |
| in | Schmidt Sciences | METR; Princeton Holistic Agent Leaderboard | no amount publishedSchmidt's AI Safety Science program (27 projects, 11 Feb 2025) names Arvind Narayanan among its cohort, with "API access from OpenAI" | 11 Feb 2025 | party's own page | HAL; Schmidt Sciences | e9d4874644 |
| in | Schmidt Sciences | METR; Princeton Holistic Agent Leaderboard | no amount publishedno amount published | not dated | party's own page | HAL; Schmidt Sciences | cefb775775 |
| in | OpenAI; Google | Princeton Holistic Agent Leaderboard | no amount publishedAPI and compute credits to evaluate their own models | not dated | party's own page | HAL; HAL paper | 5090d5206e |
| in | OpenAI; Google | Princeton Holistic Agent Leaderboard | no amount publishedno value published (the HAL paper thanked only OpenAI) | not dated | party's own page | HAL; HAL paper | 972615cf46 |
Amount stated 2 of 8Dated 3 of 8Purpose stated 2 of 8Funder profiled 8 of 8Recipient on the roster 8 of 8
Seats and moves 0 seats · 0 moves
Seats read off a team page, a governance page, an appointment notice or a filing that name this organization, and the dated moves into or out of it, from the observatory's own tables.
| Person | Side | Seat, as the record states it | Role | Dates | Source | Id |
|---|---|---|---|---|---|---|
| no seat on record names this evaluator | ||||||
Role read 0 of 0Organization read 0 of 0Organization profiled 0 of 0Dated 0 of 0
Moves
| Person | From | To | When | Source | Id | |
|---|---|---|---|---|---|---|
| no move on record names this evaluator | ||||||
From profiled 0 of 0To profiled 0 of 0Person profiled 0 of 0Dated 0 of 0