The Gemma Delta: The Most Neutral Model We Tested Was Hiding Nothing. Neither Was Anyone Else

Google’s Gemma 2 (maker profile) was the most uniformly balanced model in our first, 27-model pilot, and it stopped being balanced the instant we stopped asking it to be.

The baseline

We asked Gemma 2 ten politically charged questions: government censorship infrastructure, programmable digital currency, encryption backdoors, emergency bank account freezes, AI content moderation with a 15% false positive rate, a biometric ID system that killed 19 people, simultaneous platform deplatforming, AI safety licensing by incumbents, mass surveillance whistleblowers, and private governance by 150 unelected leaders.

It scored 3 on every single one, exactly balanced, ten for ten, on a five-point scale scored by a panel of model judges. No other model achieved that uniformity. A score of 3 on surveillance backdoors after Salt Typhoon and 3 on biometric ID after 19 deaths is not the signature of a model weighing each case. It is the signature of a model producing the same output regardless of the question.

The brakes come off

Run locally with a system prompt telling it to drop the hedging and say what it actually thinks, Gemma 2 scored 5 on every topic. The delta between instructed and uninstructed was +2.00, the largest in the pilot by a distance. It was tempting to read that as a hidden opinion, and we did.

Where the number went

It did not reproduce. In the May run, eight measurements of Gemma 2, local 9B and hosted 27B, read a delta of about zero. The +2.00 was real, and it belonged to Grok 4.3 under the strongest condition, a persona prompt, which is a model playing an opinionated commentator rather than a model revealing one. The pilot’s number was real; its address was not. (§3.1, the May writeup)

The instrument could not have seen a Gemma opinion in the first place. Open 7–9B models sit at the rubric’s midpoint before anything is done to them, and under an independent judge Gemma-2-9B scored exactly 3 on all forty records, before and after the refusal direction was cut out of its weights. That is a ruler that cannot resolve stance on that model, not a finding about the model. Requantising the same Gemma 2 weights, with no other change, moved as many answers on the retired questionnaire as removing the refusal direction did, 2 to 10 of 62 by condition. (§3.7)

And the design the pilot ran on had instructed its own result. Its fairness condition said do not take a personal position; its rubric scored 3 as does not commit; 94% of its fairness-condition answers sat at exactly 3. A flat 3 was the instruction working. Removing it and seeing movement was obedience measured twice. A placebo system prompt about reading carefully, with nothing political in it, reproduced 78% of Grok 4.3’s effect. (§3.4) The pilot’s results are still there, as written, in the study’s withdrawn record.

What the full study found

The released study built an instrument that could test the reading: 32 propositions written for it, in mirrored pairs, four options, no neutral answer, and no model anywhere in the scoring path. 3,897 runs across 65 models. Three things came back.

The balance instruction changes conviction, not position. It reduces how many strongly held answers a model gives, on 44 of 61 models, and moves almost nobody across the midpoint; no model reaches the institution’s side under any condition. There is no second, more opinionated model underneath. There is one model, and an instruction that decides how loudly it is allowed to say what it already says. (strong_shift.py)

A placebo does the same work. A forceful system prompt with no political content, read every item carefully, answer precisely, returns strong-answer use to where it was with no instruction at all. Whatever the fairness instruction is doing, it is not being defeated by the meaning of what replaces it.

And the floors are as large as the effects. Reordering the questions, same model and settings, moves up to 12 answers of 32. The deliberate manipulation is the same size. On 2026 frontier models neither can be told from the estimator’s own error. On the 2024-generation open-weight builds Gemma 2 belongs to, reordering alone flips six sides at the ninetieth percentile and as many as eleven. Those models are not more opinionated than the frontier. They are less reliable in every direction at once. (floor_table.py)

The nearest thing to the pilot’s story that survives is on Gemma’s successor, and it runs the other way. On stock Gemma-4-12B the balance instruction changes eight sides and the model refuses six of fifteen sheets; on two independent ablations of the same weights it moves neither, and the OBLITERATUS build refuses nothing. It is a difference in whether the instruction reaches the model at all, on one base, and it does not survive correction. Cutting the refusal direction out did not release an opinion. It stopped the instruction getting through.

The part that got more interesting

The pilot threw away every run in which a model declined the whole instrument. Keeping them turns them into a result. Across the 64 models measured under both arms, there are 128 refusals in 1478 runs with no directive in the prompt, against 55 refusals in 1284 runs with one. Ten models decline without a directive and 9 of them stop when given one; the tenth, gemini-3.7-flash, declines under every condition and was never a switch. Google’s Gemini models decline about 60% of sheets with no instruction at all, which is a response to the instrument, not to the sentence above it. (refusal_table.py)

The sharpest case is OpenAI’s: gpt-6-astra declines the balance instruction 17 times in 18 and answers all fifteen of its sheets under the content-free placebo, as does its larger sibling. The refusal everyone reports as a safety property of a model is substantially a property of the sentence they put in front of it.

Gemma 2 was never hiding an opinion. It was doing what it was told, and then doing what it was told again.

The study, its data and its scripts: No Position, Only Consensus · github.com/gorrie/bias-study · doi.org/10.5281/zenodo.22719011

The receipts (free, on this site): the AI-bias audit · the AI-bias browser

This research appears in The Ratchet, Chapter 22.