LMARENA (ARENA)

OLYMPUS opened an institutional file on Arena because its leaderboard is the number labs put in the first paragraph of a launch post, and since April 2025 the thing that produces the number has been a venture-backed company that also sells evaluation services to model labs. This is not a psychometric profile, since an institution has no Dark Triad. It is a mandate, a vote count, a cap table and a policy page, and the finding is the shape of the arrangement: a ranking that began as a university research project funded by gifts is now the revenue line of a company, and the labs whose models it ranks are among the customers it describes.

Institutional Archetype

THE SCOREBOARD WITH A PRIVATE ENTRANCE — Arena runs a public leaderboard built from anonymous side-by-side votes, and a separate track on which model providers test unreleased models under codenames and receive the results privately. Both tracks are documented in its own policy page. The public number is open to audit; the private track is the one that critics in 2025 argued had skewed it, and the one the company defends. Unlike the contract evaluators on the evaluator map, Arena does not grade a lab’s model on the lab’s behalf in a system card. Its output is a ranking that the lab itself then quotes. Google’s launch post for Gemini 3 said “It tops the LMArena Leaderboard with a breakthrough score of 1501 Elo” (Google), and xAI’s post for Grok 4.1 carries a section headed “LMArena Text Leaderboard” (xAI). The company has meanwhile raised $450 million in three rounds, and on 8 October 2026 added a second product beside the capability ranking: an alignment index of the same models.

Mandate & Origin

Arena’s own account: “Created by researchers from UC Berkeley, Arena (formerly LMArena) was founded to help understand AI performance in the real world. The platform, previously known as Chatbot Arena, quickly gained notoriety in the AI industry as the gold standard of evaluation. In 2025, we formed Arena Intelligence Inc. and raised funding from a16z and The University of California” (Arena, About). Its stated mission: “To measure and advance the frontier of AI for real-world use” (Arena, About). The company’s terms of use name the operator as “Arena Intelligence, Inc.” (Arena, Terms of Use).

The research project. On 1 March 2024 the LMSYS Arena team wrote: “Chatbot Arena (lmarena.ai) is an open-source project developed by members from LMSYS and UC Berkeley SkyLab,” that it “was first launched in May 2023,” and that it was then “collecting over 800,000 votes” across “more than 90 LLMs” (LMSYS, Chatbot Arena Policy). The 2024 paper that describes the platform lists Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Hao Zhang, Banghua Zhu, Michael Jordan, Joseph E. Gonzalez and Ion Stoica as authors (arXiv 2403.04132). LMSYS Corp., the nonprofit that incubated it, says it “is a 501(c)(3) non-profit,” that it “originated from a multi-university collaboration involving UC Berkeley, Stanford, UCSD, CMU, and MBZUAI in 2023,” was “incorporated as a non-profit in September 2024,” and lists Chatbot Arena among its projects with the word “(graduated)” (LMSYS, About). Ion Stoica and Joseph E. Gonzalez are among its advisors.

The company. On 23 June 2025 Arena wrote: “When we launched the first leaderboard two years ago, we weren’t trying to build a company,” and, of the platform’s origins, “LMArena started as a scrappy academic project from UC Berkeley: just a handful of PhD students and undergrads working day and night on a research prototype” (Arena, Hello from LMArena). In its seed post of 27 May 2025 the company said it had announced “about a month ago” that it was becoming a company; the post quotes Lightspeed’s Bucky Moore as “thrilled to partner with Anastasios, Wei-Lin, and Ion” and quotes Ion Stoica as “co-founder and UC Berkeley professor” (Arena, LMArena and The Future of AI Reliability). The chief executive’s own sign-off, on a June 2026 revenue post: “Anastasios CEO and co-founder” (Arena, Arena Reaches $100M in 8 Months). TechCrunch describes the origin as “an open research project built by UC Berkeley researchers Anastasios Angelopoulos and Wei-Lin Chiang, and was originally funded through grants and donations” (TechCrunch, 6 Jan 2026).

The name. On 28 January 2026: “Now that our platform has become the de facto Arena, we have outgrown the ‘LM,’ and it’s time to drop it” (Arena, LMArena is now Arena). The company moved to arena.ai the same day. lmarena.ai now redirects to arena.ai.

Funding & Backers

In Arena’s words. The seed round, 27 May 2025: “we’ve raised $100M in seed funding”; “The seed round was led by a16z and UC Investments (University of California) with participation from Lightspeed, Laude Ventures, Felicis, Kleiner Perkins, The House Fund, and others.” Anjney Midha of a16z: “We invested in LMArena because the future of AI is about reliability and it needs rigorous, innovative, transparent evaluation to get there.” Jagdeep Singh Bachher, chief investment officer of UC Investments: “Supporting innovation from university labs such as those at UC Berkeley is essential for building technologies that responsibly serve the public and advance the field” (Arena, seed post). The Series A, 6 January 2026: “We’ve raised $150M of Series A funding led by Felicis and UC Investments (University of California), with participation from Andreessen Horowitz, The House Fund, LDVP, Kleiner Perkins, Lightspeed Venture Partners and Laude Ventures” (Arena, Series A). The Series B, 8 October 2026: “Our Series B is co-led by Lightspeed Venture Partners and Khosla Ventures with participation from Salesforce Ventures, 01 Advisors, Dell Technologies Capital, Acrew Capital, and Endeavor Catalyst. We’re also excited to have support from our existing investors a16z, Felicis, AMP PBC, QuantumLight, The House Fund, Panos Madamopoulos, and others” (Arena, Series B).

• Valuations. Arena’s posts state one: “a $200 million Series B at a $3.1 billion valuation” (Arena, Series B). TechCrunch reports the other two: the Series A “at a post-money valuation of $1.7 billion,” and the seed “at a $600 million valuation” (TechCrunch). The About page still says “With $250 million in venture funding,” the sum of the seed and Series A (Arena, About).
• Revenue, in Arena’s words. “We’ve exceeded $100M in annualized revenue” (Arena, Series B); earlier, “Arena has crossed $100M annualized revenue run rate within eight months of launching our enterprise offering” (Arena, 29 Jun 2026). The product is AI Evaluations, launched 16 September 2025: “This service offers enterprises, model labs, and developers comprehensive evaluation services grounded in real-world human feedback” (Arena, New Product: AI Evaluations). TechCrunch reported an annualized “consumption rate” of “$30 million as of December” 2025 (TechCrunch). No Arena page names a customer or states revenue by customer type.
• What it said before. The March 2024 policy FAQ asked “Who will fund this effort? Any conflict of interests?” and answered: “Chatbot Arena is only funded by gifts, in money, cloud credits, or API credits. The gifts have no strings attached” (LMSYS, Chatbot Arena Policy). LMSYS’s current sponsor list, for its own donations, includes a16z, Google Cloud, NVIDIA, Hugging Face, Together and Kaggle (LMSYS, About); a16z is also an Arena investor in all three rounds.
• Commitments attached to the money. The seed post: “commercial sustainability should never come at the cost of community trust or great science,” and Arena will provide “Transparent, science-driven leaderboard policies that ensure every model is tested fairly, consistently, and with community input” and “a neutral, trustworthy platform without exclusivity.” The AI Evaluations post lists “Neutrality. All models—commercial, open source, public, or awaiting release—are evaluated using the same community-driven methodology, with no preferential treatment.”

Institutional Voice & Intent

Arena on its purpose, 8 October 2026: “This round is a vote of confidence in the idea that as AI gets more powerful, the world needs an independent, data-driven approach to measure not just the capabilities of AI, but whether it can be trusted and used safely in the real world,” and “The world needs a neutral third party to measure how safe and aligned AI actually is once it’s in the hands of real people. Arena is stepping into that role today” (Arena, Series B). On its method, the 2025 introduction: “The arena is designed like a tournament where models are compared anonymously side by side and users vote for the better response,” and “around 70% of prompts each month are fresh” (Arena, Hello from LMArena). On what its own investors and labs get: “We also share a percentage of the collected data back with model providers to help them improve their models from our community’s perspective” (same page).

Stated intent: Be “a neutral third party” that measures AI “capabilities” and now “whether it can be trusted,” funded by a business that never gives preferential treatment to any model.

Observed intent: Turn a free crowd leaderboard into the layer labs test against before release and quote after it. The policy page lets providers test unreleased models anonymously and “Share the results privately with the model provider” (Arena, Leaderboard Policy); the leaderboard shows them only after public release. The company sells paid evaluation to “model labs” among others (Arena, AI Evaluations), and its newest product, the Alignment Index, scores the same labs’ models on “Unauthorized Action,” “False Attribution” and “Deceptive Completion,” signals it says are “inspired by the same definitions that labs like OpenAI and Anthropic have published in their system cards” (Arena, Series B).

Gap: The company states its neutrality and publishes the rules, the ranking code and a share of the data. What its pages do not settle is the weight of the private track and of the paid service in the ranking and in the revenue: no page gives the number of providers using pre-release testing, the number of variants any one provider tested, or the share of revenue that comes from labs it ranks. The critics’ claim in 2025 was that the private track skewed the board; Arena’s reply was that the skew vanishes with fresh votes and that every provider may use it. Both are on the record, and the figures that would let a reader weigh them are the ones neither page prints.

Interest: Arena argues for crowd preference as the measure of models, for pre-release testing open to all providers, and for a neutral third party to grade alignment. What each position would pay is set out from Arena’s own pages; no contract value is published.

• Human preference votes as the measure. Position, Arena’s: “human preference will remain a core metric to AI progress” (Arena, seed post). What it pays: the votes are the asset the paid service and the reward model are built from. AutoEval, announced 30 July 2026, trains “a reward model on Arena’s large-scale human preference dataset, consisting of millions of pairwise comparisons from live votes” and uses it “to cast votes automatically” (Arena, AutoEval); Lightspeed’s Bucky Moore calls the dataset “one of the most valuable data assets in AI” (Arena, seed post). Best case, Arena’s: “Since there’s no payment or external incentive, votes come from intrinsic motivation” (Arena, Hello from LMArena). Who else gains: the labs, which receive “a percentage of the collected data” and a rank to cite, and the investors, whose stake is in the company that holds it (Arena, Hello from LMArena).
• Pre-release testing open to every provider. Position, the 2025 response: “any model provider can submit as many public and private variants as they would like, as long as we have capacity for it” (Arena, Our Response), and the policy page: “Model providers are allowed to test multiple variants of their models before making them public, subject to our system’s constraints” (Arena, Leaderboard Policy). What it pays: private testing is a service labs use and Arena runs; the policy page says “we share conversation data with model providers to help them improve their models” (Arena, Leaderboard Policy). Best case, the response’s: “Boosts in a model’s score due to pre-release testing are minimal,” “around +11 Elo after 50 tests and 3000 votes” (Arena, Our Response). Who else gains: the larger labs, which the paper counts as the heaviest private testers (Meta 27 variants in one month, Google 10), and by Arena’s own count Cohere, with 9 pre-release models “2-3x more pre-release tests than labs like xAI/OpenAI” (arXiv HTML; Arena, Our Response).
• A neutral third party for alignment. Position, Series B: “The world needs a neutral third party to measure how safe and aligned AI actually is” (Arena, Series B). What it pays: a second leaderboard, announced the day of a $200 million round, in a market where labs publish their own alignment results in system cards. Best case, Arena’s: “static benchmarks break down once models recognize they’re being tested” (Arena, Series B). Who else gains: the other evaluators on the map, among them METR, Gray Swan and Epoch AI, whose case for independent measurement is the same, and the investors the post names.

Interest is a documented fact about each position, not a claim about motive, and not a verdict that the argument is wrong. Each best case is stated at the strength its author states it.

Position in the Apparatus

Arena sits in the ranking layer, not the audit layer. Scale AI and Epoch AI build benchmarks; Gray Swan and Irregular test models for labs; Stanford CRFM keeps HELM, now in maintenance mode. Arena runs the board the labs quote. Artificial Analysis runs the fixed-task counterpart, an index the labs also quote at launch. The AI evaluators map lists it beside the contract evaluators.

• The labs cite the ranking in launch posts. Google: “Gemini 2.5 Pro topping LMArena for over six months,” and Gemini 3 “tops the LMArena Leaderboard with a breakthrough score of 1501 Elo” and “tops the WebDev Arena leaderboard by scoring an impressive 1487 Elo” (Google). xAI on Grok 4.1: “In LMArena’s Text Arena, Grok 4.1 Thinking (code name: quasarflux) holds the #1 overall position with 1483 Elo —a commanding margin of 31 points over the highest non-xAI model,” and, (xAI, Grok 4.1); and, of an earlier model, “An early version of Grok-2 has been tested on the LMSYS leaderboard under the name “sus-column-r”” (xAI, Grok-2). Both posts show the private track in use: a codename tested before release and quoted after it.
• The critique. “The Leaderboard Illusion,” posted to arXiv on 29 April 2025 by Shivalika Singh and twelve co-authors, among them Sara Hooker and Sayash Kapoor, with affiliations listed as Cohere Labs, Princeton, Stanford, the Allen Institute for AI and the University of Washington, among others: “we identify systematic issues that have resulted in a distorted playing field. We find that undisclosed private testing practices benefit a handful of providers who are able to test multiple variants before public release and retract scores if desired” (arXiv 2504.20879). It counts “27 private LLM variants tested by Meta in the lead-up to the Llama-4 release,” says “Providers like Google and OpenAI have received an estimated 19.2% and 20.4% of all data on the arena, respectively,” and “a combined 83 open-weight models have only received an estimated 29.7% of the total data” (arXiv 2504.20879). Its recommendations include “Prohibit score retraction after submission” and “maximum 3 model variants tested concurrently per provider” (paper, HTML).
• The response. Arena, 9 May 2025: “We are grateful for the feedback and have plans to improve Chatbot Arena as a result of our ongoing discussions with the authors.” It promised that “In a future policy release, we will explicitly state that model providers are all allowed to test multiple variants of their models pre-release,” to “mark which models are retired,” and to label scores “provisional” until “additional 2,000 fresh votes have been collected after model release, if more than 10 models were pre-release tested in parallel.” It called other claims wrong: “Official Chatbot Arena stats (published 2025/4/27) show Open Models at 40.9%,” the 100-point figure “is a simulation,” “There’s no secret or unstated policy,” and “We stand by the integrity and transparency of the Chatbot Arena platform” (Arena, Our Response). TechCrunch records the dispute as an allegation “LMArena has vehemently denied” (TechCrunch).
• The policy as it now reads. Last updated 1 September 2026: “The leaderboard will only include first-party foundation models that are generally available to the global public”; an unreleased model’s score is “preliminary” until fresh votes arrive; for early release on Arena the provider “must confirm in writing that the pre-release model is identical to the model they intend to release publicly”; “Model providers are allowed to test multiple variants of their models before making them public, subject to our system’s constraints”; “In every battle, at least one of the models is a publicly available model. At least 20% of battles will be between publicly available models only”; “The regression for computing Arena scores uses reweighting, such that no matter how the sampling probabilities are set, the Arena scores remain unbiased”; and “The model evaluation and ranking pipelines have been open sourced in the Arena-Rank repository” (Arena, Leaderboard Policy). The original 2024 policy had no multiple-variant sentence, no 30-day access rule and no written-identity confirmation (LMSYS, 2024 policy).
• The user licence. Arena’s terms give the company a “non-exclusive, transferable, perpetual, irrevocable, worldwide, fully-paid, royalty-free, sublicensable” licence to what users submit, including the right to “derive revenue or other remuneration from Your Content” (Arena, Terms of Use). The voters who produce the ranking are the licensors.
• Hugging Face’s retired board. A Hub post by user burtenshaw, 14 March 2025, opens “The open LLM leaderboard is completed, retired, dead,” and lists community boards: “Chatbot Arena LLM Leaderboard is the first port of call for checking out the best model” (Hugging Face, post). See Hugging Face. The post is one user’s list, not a statement by the company that Arena succeeds the board.
• Alignment and agents, 2026. Arena counts “7 million sessions in Agent Arena in less than 5 months since launch.” The Alignment Index “starts narrow and is evidence-based by design” and shares “the methodology and initial results for 20+ frontier models”; Arena’s own summary of the index says it compares “27 models across 90,000 real-world agent sessions” (Arena, Series B; Arena, Arena Alignment Index).

Actions & Leadership Choices

CONDUCT: WATCH — A COMMERCIAL SERVICE SOLD TO THE MODEL LABS THE LEADERBOARD RANKS. No wrongdoing is alleged. The documented facts: the leaderboard began as a gift-funded university project that said gifts had no strings, became a company that has raised $450 million, sells evaluation to model labs, runs a pre-release testing track that it says is open to all, and adds a second leaderboard for the same labs’ alignment. The structural point is the one every paid evaluator shares, sharpened by the ranking’s visibility: the labs it scores quote it, test on it privately and may buy services from the company that runs it, and no page gives the weights of those flows.

• 1 March 2024. The LMSYS Arena team publishes the Chatbot Arena policy, with the “gifts have no strings” funding answer (LMSYS).
• September 2024. LMSYS Corp. incorporated as a nonprofit (LMSYS, About).
• April 2025. Arena announces it is becoming a company (Arena, seed post, “about a month ago”).
• 29 April 2025. “The Leaderboard Illusion” posted to arXiv (arXiv).
• 9 May 2025. Arena publishes its response (Arena).
• 27 May 2025. $100 million seed announced, relaunch of the site (Arena).
• 16 September 2025. AI Evaluations launched (Arena).
• 6 January 2026. $150 million Series A (Arena).
• 28 January 2026. LMArena becomes Arena, moves to arena.ai (Arena).
• 29 June 2026. $100 million annualized revenue announced (Arena).
• 30 July 2026. AutoEval scores added to the leaderboards (Arena).
• 1 September 2026. Policy page last updated (Arena).
• 8 October 2026. $200 million Series B at $3.1 billion and the Arena Alignment Index (Arena).

Alliance Map

These run at the same time. One company is funded by venture firms, by the university system where it began and by the corporate venture arms of Salesforce and Dell, sells evaluation to model labs, is quoted by two of those labs in launch posts, runs a private testing track for them, and in October 2026 started grading their alignment. Each tie is stated by Arena, a lab or LMSYS. The overlap is the finding.

Reach Assessment

Institutional: Labs write the rank into launch posts, test unreleased models on it under codenames and, by Arena’s account, receive data from it; enterprises can buy evaluation from it. No statute or procurement rule names it. Memetic: High: ten million monthly visitors, 82 million votes and a score that a model announcement can lead with. Civilizational: If a vote count decides which model the world believes is best, the company that counts the votes sits between the labs and the public they sell to; if it now also scores alignment, the same company sits between the labs and their own safety claims. The record shows a published policy, open ranking code, a shared dataset and a public dispute with its critics. It does not show the weights: how much of the revenue comes from the labs it ranks, and how many variants each lab tested. Those two numbers are the gap.

Score basis. Atk 8 is reach: Google’s Gemini 3 launch post says “It tops the LMArena Leaderboard with a breakthrough score of 1501 Elo,” xAI’s Grok 4.1 post carries an “LMArena Text Leaderboard” section, Arena reports “10M+ monthly visitors” and “82M+ total votes,” and unreleased models are tested on it under codenames (Google; xAI; Arena, 29 Jun 2026). The points under 9 are that no statute, procurement rule or government pre-deployment test names it and a lab can stop citing it. Peers on atk 8: Gray Swan, Meridian Labs and Scale AI; Hugging Face holds 5 because its reference leaderboard is retired, and US CAISI and the UK AI Security Institute hold 9. Def 4 is funding insulation: Arena sells “evaluation services” to “enterprises, model labs, and developers” (Arena, AI Evaluations), its Series B was co-led by Lightspeed and Khosla with Salesforce Ventures and Dell Technologies Capital participating (Arena, Series B), and in 2024 its predecessor wrote that it “is only funded by gifts” (LMSYS); the points off are that no customer or amount is named. The points kept are that every investor is named round by round, the policy and ranking code are public, and no page read names a lab as an investor. Peers on def: Promptfoo holds 3 because a lab owns it, Meridian Labs and Andon Labs hold 5, and Scale AI, Gray Swan and Hugging Face hold 6. Hp 7 is entrenchment: “350 million sessions across the entire Arena platform,” “62 million votes,” “We’ve exceeded $100M in annualized revenue” and a $3.1 billion valuation after three rounds in seventeen months, with Google and xAI quoting the board (Arena, Series B); against that, no statute installs it and labs can submit elsewhere, and Hugging Face’s post that names it first is one user’s list. Peers on hp 7: Epoch AI and Meridian Labs; Hugging Face and Scale AI hold 8. Hazard 63 follows from 8/4/7 at the house formula and sits level with Hugging Face (62) and Epoch AI (64), whose reach is lower and insulation higher.

Sources: Arena — About; Arena — Blog; Arena Alignment Index — Arena (8 Oct 2026); LMArena and The Future of AI Reliability — Arena (27 May 2025); Fueling the World’s Most Trusted AI Evaluation Platform — Arena (6 Jan 2026); Arena’s $200 Million Series B — Arena (8 Oct 2026); New Product: AI Evaluations — Arena; Arena Reaches $100M in 8 Months — Arena; LMArena is now Arena — Arena; Introducing AutoEval — Arena; Arena Leaderboard Policy — Arena; Our Response to ‘The Leaderboard Illusion’ Writeup — Arena; Hello from LMArena — Arena; Arena — Terms of Use; LMSYS — About; Chatbot Arena Policy — LMSYS (1 Mar 2024); The Leaderboard Illusion — arXiv; The Leaderboard Illusion — arXiv HTML; Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference — arXiv; Gemini 3 — Google; Grok 4.1 — xAI; Grok-2 — xAI; LMArena lands $1.7B valuation — TechCrunch (6 Jan 2026); Post by burtenshaw — Hugging Face.