VALS AI

OLYMPUS opened an institutional file on Vals AI because its Finance Agent benchmark is the number Anthropic printed in a product launch, because its Vals Index now ranks every frontier model on a GDP-weighted composite, and because the company says in the same breath that it is “the independent evaluator of artificial intelligence” and that its revenue comes from the AI industry and, since July 2026, from government. This is not a psychometric profile, since an institution has no Dark Triad. It is a mandate page, a funding announcement, a government-collaborations page, a methodology page and two news interviews, and the finding is the shape of the arrangement: a two-year-old company whose benchmark leaderboard is free to read, whose private test sets and evaluation services are sold, whose lead investor also leads the funding of its nearest crowd-vote rival, and whose government pages sit beside a statement about independence.

Institutional Archetype

THE INDEPENDENT EVALUATOR ON THE PAYROLL OF THE EVALUATED — Vals tests every model it can reach on tasks drawn from professions, publishes the results, and is paid by parties in the market it measures. Its own sentence for what it is: “We started Vals AI to solve this problem, as the independent evaluator of artificial intelligence” (Vals AI, About). Its stated method: “Scores are based on our privately held test sets to preserve the integrity and signal of our results” (same page). Its own claim of reach: “Our results have been cited in model cards from OpenAI, Anthropic, Google, Meta, and xAI” (Vals AI, Series A). Where Artificial Analysis publishes a composite of general capability and LMArena counts crowd votes, Vals builds task suites out of professional work, with partner institutions in each field, and keeps the test sets private. Where Epoch AI is philanthropy-funded and Dreadnode sells the range it grades on, Vals is venture-funded and says its clients pay it to test their models.

Mandate & Origin

Vals’s account of its scope: “We build benchmarks that measure the ability of models to do the work of lawyers, bankers, engineers, and doctors,” and on its results, “These benchmarks, including Finance Agent, have rapidly become industry standards” (Vals AI, About). Its stated reason for existing: “New benchmarks are often produced or run by the same companies that build the models” (Vals AI, Series A). The benchmarks page lists legal, finance, healthcare, mental health, math, science, education, voice, coding, cyber, games and social-mobility sections, with each benchmark tagged Proprietary, Industry Partner or Academic (Vals AI, Benchmarks).

Founders and people. The company’s own About page names no founder, title or team member. Its Series A post is signed by Rayan Krishnan and speaks of “what we predicted when we started Vals” (Vals AI, Series A). TechCrunch, 19 September 2026, calls Krishnan “the company’s 25-year-old co-founder” who “previously interned at Palantir, and, as an undergraduate at Stanford, worked for Microsoft and the school’s much-lauded artificial intelligence lab” (TechCrunch, as reported). CNBC, 11 October 2026, calls him “CEO of independent evaluator Vals AI” (CNBC, as reported). The Finance Agent Benchmark paper (arXiv 2508.00828, submitted 20 May 2025) lists Antoine Bigeard, Langston Nashold, Rayan Krishnan and Shirley Wu as authors, three of them with the affiliation “Vals AI, Inc.” and one with Stanford University ([arXiv](https://arxiv.org/abs/2508.00828; arXiv HTML). The July 2026 public-sector memo, signed by Krishnan and Glenn Parham, says “we’ve brought on Glenn Parham as head of Public Sector” and that “He’s spent his career bringing AI into national security missions at the Pentagon and in big tech” (Vals AI, Public Sector memo).

Legal form and place. The paper’s affiliation line is “Vals AI, Inc.” (arXiv HTML); the paper states “This report was funded by Vals AI, a startup dedicated to evaluating Large Language Models” (same page). TechCrunch describes a startup “formed in 2024” with a two-floor office on San Francisco’s Folsom Street (TechCrunch, as reported). The memo says Vals “opened an office in Washington, DC” (Vals AI, Public Sector memo).

Funding & Backers

In Vals’s words. “Today we’re announcing our $40M Series A at a $400M valuation, led by a16z, with participation from existing investors 8VC, Pear VC, and BloombergBeta and new investors HRT Ventures and Next Ladder Ventures” (Vals AI, Series A, 13 August 2026). TechCrunch adds that “last year, it managed to secure a seed round led by 8VC and Bloomberg Beta” (TechCrunch, as reported). No Vals page gives the seed amount, a per-investor amount or a named individual backer.

• Growth, in its words. “Our revenue has grown 8x compared with all of 2025, our customer base has doubled, and our incredible team has tripled in six months” (Vals AI, Series A). TechCrunch: “Vals, which started the year with only eight people, has already tripled to a team of 25”; CNBC: Krishnan said the startup “has grown from eight employees to roughly 30 this year” (TechCrunch; CNBC, both as reported). No revenue figure is published.
• How it earns. TechCrunch, reporting the interview: “Companies pay Vals to test their models” (TechCrunch, as reported). Vals’s methodology page describes “A larger, privately held dataset, which we license to companies for their own internal validation” (Vals AI, Methodology). The Series A post: “Some of the biggest enterprise AI deployments use Vals to choose which models to build on and measure their products against the frontier” (Vals AI, Series A). The memo: “We work closely with the leading AI labs to keep up with the frontier while remaining independent” (Vals AI, Public Sector memo). No page names a paying lab, enterprise or price.
• Who funds what, by name. No model developer is named among the investors. Of the named funds, a16z is also the lead investor in Arena’s seed round and an investor in xAI, per those profiles (Arena; xAI). The eight individuals Artificial Analysis lists as backers (Artificial Analysis) and the individual backers Mechanize names (Mechanize) do not appear among Vals’s named investors.

Institutional Voice & Intent

The Series A post on the market: “Labs lack a credible way to demonstrate continued model progress. Enterprises increasingly see pressure to adopt and spend on AI without the means to quantify ROI. Governments must develop their own expertise to measure frontier capabilities, cyber risk, and the pace of global competition,” and “AI has quickly become a trillion-dollar market without the independent measurement institutions that other markets of this scale, like finance and healthcare, require” (Vals AI, Series A). Krishnan, in TechCrunch’s report, compares the revenue model to a student paying the College Board to take the SAT and expects AI companies to “submit public filings” with benchmarks and evaluations like Vals’s (TechCrunch, as reported).

Stated intent: Be “the independent evaluator of artificial intelligence,” with private test sets that cannot be trained on and benchmarks built with reference institutions in each field.

Observed intent: Turn a benchmark site into the measuring layer that labs, enterprises and governments all buy from. The company runs an index across finance, coding, legal and tax; opened a Public Sector line and a Washington office; and describes work for the Department of Commerce and Congress. Its government page states: “Trusted and cited by leading AI labs. Bringing that same independent rigor to the public sector” (Vals AI, Government).

Gap: Vals states that it is independent, publishes a methodology with error bars, and runs the same harness on every model it tests. What its pages do not settle is who the paying clients are, how much of its revenue comes from AI developers whose models it ranks, and whether any lab has seen results before they were public. The one first-hand lab citation found is Anthropic’s, and Anthropic’s current model pages do not mention Vals. Both statements are on the record; the figure that would let a reader weigh them is the one neither prints.

Interest: Vals argues for independent third-party evaluation, for private test sets, and for government use of outside benchmarks. What each would pay is set out from its own pages; no contract value is published.

• Independent evaluation as the product. Position: “AI has quickly become a trillion-dollar market without the independent measurement institutions that other markets of this scale… require” (Vals AI, Series A). What it pays: the independence claim is what clients, investors and agencies are asked to rely on, and a lab that quotes a Vals number widens the audience for the paid services. Best case, Vals’s: when “New benchmarks are often produced or run by the same companies that build the models,” an outside measurer has to exist. Who else gains: labs that top a Vals leaderboard and print it, and enterprises that buy a vendor-neutral selection.
• Private test sets. Position: “Scores are based on our privately held test sets to preserve the integrity and signal of our results” (Vals AI, About). What it pays: a private test set cannot be copied, so reproducing the score requires paying Vals, and the larger private validation set is licensed to companies (methodology page, above). Best case: public test sets leak into training data. Who else gains: labs, which can cite a score on a set they cannot see, and companies that license the private validation set for their own internal testing.
• Government evaluation by outside benchmark. Position, the memo: “Government needs objective assurances, not the promises of vendors” (Vals AI, Public Sector memo). What it pays: Vals Public Sector is a line of business with a head hired from the Pentagon and big tech and an office in Washington, DC; its government page lists work with NIST for CAISI and with CNAS and Virginia Tech’s National Security Institute. Best case: officials “asked to accept unknown risks” need independent evidence. Who else gains: CAISI, which receives benchmarks and data from an outside vendor, and the labs, whose models are tested by a vendor that works closely with them.

Interest is a documented fact about each position, not a claim about motive, and not a verdict that the argument is wrong. Each best case is stated at the strength its author states it.

Position in the Apparatus

Vals sits in the ranking layer beside Artificial Analysis and LMArena, with a benchmark-builder’s habit: it partners with an institution in each field, builds a task taxonomy, and keeps the test set private. It is not in the audit layer with METR and Irregular, which test models inside release decisions, though its newer frontier-risk lines (the RSI Index, a cyber benchmark, mental-health and child-safety work) approach it. Epoch AI and Scale AI build benchmarks too. The AI evaluators map does not list it.

• A lab cites the benchmark in launches. Anthropic, 15 July 2025: “Claude 4 models outperform other frontier models as research agents across financial tasks in Vals AI’s Finance Agent benchmark” (Anthropic, Claude for Financial Services). Anthropic, 29 September 2025, in the Sonnet 4.5 launch footnotes: “All scores reported were run and published by Vals AI on their public leaderboard,” with OpenAI and Gemini scores read from “public Vals AI leaderboard” (Anthropic, Claude Sonnet 4.5). Anthropic, 5 May 2026: Claude Opus 4.7 “leads the industry on Vals AI’s Finance Agent benchmark, at 64.37%” (Anthropic, Agents for financial services). Anthropic’s launch pages for Claude Opus 5, Opus 5.5 and Sonnet 5.5 do not contain the name Vals. OpenAI’s GPT-5.6 and GPT-6 Astra pages and Meta’s Muse Spark Open model card do not contain it either; Vals’s own claim that OpenAI, Google, Meta and xAI model cards cite it is not confirmed from those pages.
• The company runs the labs’ models and publishes how. For Alibaba’s Qwen 3.6 27B it writes that “All evals were run via the Alibaba API” at the vendor’s recommended settings; the home page carries a “Congrats to the Anthropic team on the release!” line under its Claude evaluations (Vals AI, home). Its methodology page says scores include “rubric-based LLM-as-a-judge accuracy metrics” (Vals AI, Methodology).
• It audits cheating by the models it ranks. Vals publishes “AI Cheating is on the Rise,” an integrity audit across BioMysteryBench, Terminal-Bench 2.1 and SWE-bench Verified (15 September 2026), and a study of Xiaomi’s MiMo v2.6 coding tasks whose fixes are readable in Git history (7 October 2026) (Vals AI, Blog).
• Industry partners sit inside the benchmarks. Harvey’s Legal Agent Benchmark appears on Vals’s page tagged “Industry Partner” and described as testing “an agent’s ability to complete legal work using documents, spreadsheets, presentations, and file-system tools” (Vals AI, Benchmarks). Vals’s 27 February 2025 legal AI report, built on the lawyer-baseline method, says “Harvey opted into six out of seven tasks” (Vals AI, Legal AI Report). For the Web Search Index, Vals announced “Exa, Keenable, Parallel, and Tavily are joining us as partners on the Vals Web Search Index” (Vals AI, home); the RSI Index lists CoreWeave under “Partners in Evaluation” (Vals AI, RSI Index).
• The index is a weighted composite. The Vals Index is “a single measure of AI’s potential economic impact — agentic model performance across finance, coding, legal, and tax tasks, weighted by each sector’s share of U.S. GDP” (Vals AI, Vals Index); the benchmarks page shows it updated 16 October 2026 with 49 models tested.

Actions & Leadership Choices

CONDUCT: NO CONDUCT ISSUE ON THE RECORD — LAB CLIENTS UNNAMED, A VC LEAD INVESTOR SHARED WITH ARENA AND xAI, GOVERNMENT WORK SOLD ON THE SAME INDEPENDENCE CLAIM. No wrongdoing is alleged. The documented facts: a benchmark startup formed in 2024, a seed round led by 8VC and Bloomberg Beta, a Finance Agent benchmark that Anthropic cited in three launches, a Series A led by a16z, paid testing of models for companies, an index across four sectors, and a government line that began in July 2026 with a former Pentagon and big-tech hire. The structural point is the one every paid evaluator shares: the labs it scores may quote it and may buy services from it, and no page gives the weights of those flows.

• 27 February 2025. Vals’s legal AI report compares CoCounsel, Vincent AI, Harvey Assistant and Oliver with a lawyer baseline (Vals AI).
• 20 May 2025. The Finance Agent Benchmark paper, with a “funded by Vals AI” statement (arXiv).
• 15 July 2025. Anthropic cites the Finance Agent benchmark in Claude for Financial Services (Anthropic).
• 29 September 2025. Anthropic’s Sonnet 4.5 table credits Vals for the Finance Agent scores (Anthropic).
• 5 May 2026. Anthropic’s finance-agents post gives Opus 4.7 the lead at 64.37% (Anthropic).
• 2 July 2026. Vals Public Sector announced, with Glenn Parham as head and a Washington, DC office (Vals AI).
• 27 July 2026. Partnership with CNAS and Virginia Tech’s National Security Institute on evaluating military AI (Vals AI, Government).
• 28 July 2026. Vals states it is providing benchmarks and data to NIST for CAISI evaluations (Vals AI, Government).
• 13 August 2026. Series A announced with Vals Smith, Frontier Risk Benchmarks (RSI Index with CoreWeave, a cyber benchmark, mental-health work) and Vals 2.0 (Vals AI, Series A).
• 23 September 2026. First Public Sector Summit at STATION DC (Vals AI, Government).
• 16 October 2026. Vals Science Index added; the Vals Index lists 49 models (Vals AI, Benchmarks).

Alliance Map

These run at the same time. One venture-funded company is paid by companies to test their models, is cited by Anthropic in launch material, hosts an industry partner’s benchmark inside its legal suite, lists a cloud company as an RSI Index partner and four search vendors as Web Search Index partners, supplies benchmarks to the agency that tests the labs before release, and partners with a security think tank on military-AI evaluation. Its lead investor also leads the seed round of its crowd-vote rival and invests in a lab it ranks. The seats link outward: Chris Painter of METR is an adjunct senior fellow at CNAS, and Paul Christiano is a senior technical advisor at CAISI, per the evaluator interlocks; neither is documented as a party to any Vals arrangement. The overlap is the finding.

Reach Assessment

Institutional: One lab prints the Finance Agent number in launch material, enterprises and agencies are sold the testing, and no statute, procurement rule or required pre-release test names it. Memetic: Moderate: a Vals ranking is a number a launch can cite and trade press repeats, and the company’s own pages claim wider lab use than the lab pages checked show. Civilizational: If the Vals Index becomes the measure of how much work a model can do for the economy, the company that sets the sector weights and holds the test sets sits between the labs and the buyers, and between the labs and the agency that tests them; if it is also paid by the labs for testing, it stands on both sides of the number. The record shows a stated independence claim, private test sets, error bars, a methodology page and cheating audits of the labs’ own models. It does not show who pays or how much. Those are the gap.

Score basis. Atk 6 is reach: Anthropic printed the Finance Agent result in a July 2025 product launch, a September 2025 model launch table (“run and published by Vals AI”) and a May 2026 agents launch, and Vals states it supplies benchmarks to NIST for CAISI’s evaluations (Anthropic; Anthropic; Vals AI, Government). The points under 8 are that no statute or required pre-release test names it, that the only first-hand lab citation found is Anthropic’s, and that Anthropic’s Opus 5, Opus 5.5 and Sonnet 5.5 launch pages do not mention it. Peers on atk 6: none identical; Artificial Analysis and LMArena hold 8 because several labs print their numbers; Epoch AI, Dreadnode and Gryphon Scientific hold 5. Def 4 is funding insulation: TechCrunch reports “Companies pay Vals to test their models,” the methodology page licenses a private validation set to companies, the Series A names six funds and no customer, and a16z leads the Series A while also leading Arena’s seed and investing in xAI (TechCrunch; Vals AI, Methodology; Vals AI, Series A). The points kept are that no lab appears among the investors, that the independence claim is stated and the methodology published, and that the company reports cheating by the models it ranks. Peers on def 4: Artificial Analysis and LMArena, which also sell services to the labs they rank; Dreadnode and Gryphon Scientific hold 5 and 4, Epoch AI holds 7 on philanthropic funding. Hp 5 is entrenchment: a GDP-weighted index of 49 models, private test sets, a benchmark one lab has cited since July 2025, revenue stated as 8x last year and a government line (Vals AI, Series A; Vals AI, Benchmarks); against that, it is two years old, about 30 staff by its CEO’s account, no statute installs it, no multi-year contract is disclosed and any lab can stop citing it. Peers on hp 5: none identical; Artificial Analysis holds 6, LMArena and Epoch AI hold 7, Dreadnode holds 4 and Gryphon Scientific 3. Hazard 50 follows from 6/4/5 at the house formula and sits between Dreadnode (47) and Artificial Analysis (60).

Sources: Vals AI — About; Vals AI — Series A: Always a Higher Peak (13 Aug 2026); Vals AI — Public Sector memo (2 Jul 2026); Vals AI — Government; Vals AI — Methodology; Vals AI — Benchmarks; Vals AI — Vals Index; Vals AI — RSI Index; Vals AI — Legal AI Report; Vals AI — Blog; Vals AI; Finance Agent Benchmark — arXiv 2508.00828; arXiv HTML; Agents for financial services — Anthropic (5 May 2026); Claude for Financial Services — Anthropic (15 Jul 2025); Introducing Claude Sonnet 4.5 — Anthropic (29 Sep 2025); Vals, backed by Andreessen Horowitz — TechCrunch (19 Sep 2026); AI’s quiet safety gatekeepers — CNBC (11 Oct 2026).