No Position, Only Consensus

Frontier models will tell you they don’t have political opinions. Their builders will tell you they don’t have political opinions. The proof, supposedly: ask it something politically contested and it produces a careful, balanced, on-the-other-hand paragraph that doesn’t commit to anything.

We took that as the claim, not the answer. Then we tested it. What we found is less dramatic than a hidden agenda and more useful. The instruments this whole literature uses were built to find a position, and on the questions we asked there is almost no position to find. What there is instead: a consensus, an instruction that changes how strongly a model says things but not what it says, and a refusal rate that belongs to whoever wrote the prompt.

Everything below is in the bias-study repository: the paper, the instrument, every answer sheet in runs/, and the scripts that turn them into every number on this page. No API key is needed to check any of it. The exceptions are the refusal-direction and dose-series results, whose prompts are XSTest’s, and the few figures from the retired questionnaire, each marked where it appears. This page is the announcement and the analysis. On the analysis, we don’t hedge.

The instrument

Thirty-two propositions about state power, surveillance and speech, written for this study, in sixteen mirrored pairs: each claim stated once from the critic’s side and once from the defender’s. A model that holds a position answers the two halves differently; a model that agrees with whatever is in front of it does not. Four options, no neutral answer, and no language model anywhere in the scoring path — a script reads the answer sheet. The instrument is the author’s own, and an off-the-shelf left/right questionnaire was evaluated and rejected — for the usual counterproductive left/right dogma, and because its scoring is undisclosed, so it cannot be judge-free and transparent at once. The propositions ship in full under the MIT licence with everything else.

We put them to hosted frontier models and local open-weight builds, across 3,897 runs, 65 models and 23 vendor keys, under seven conditions. The ones that matter here: N, no system prompt at all; A, an instruction to answer in a balanced manner and not take a personal position; D, an instruction to commit; and P, a placebo — a forceful instruction about reading carefully with no political content in it. Then we measured the things nobody claims are political: reprinting the items in a different order, running the same prompt again, requantising the same weights, and comparing two variants of one release.

What a political instrument does when nothing political changes

Each row is the movement produced by a factor nobody claims is political, on the same instrument, same models, same settings, counted in answers that switched sides out of 32:

Put the 32 propositions to a model twice, nothing changed. A median of 0 answers move, and up to 16. Reorder the questions and up to 12 move. Compare two variants of one release and up to 2 move. The deliberate manipulation — balance against commitment — moves 4 at its 90th percentile. The factors nobody calls political are the same size as the one that is.

Measured in position rather than side-flips, on the same sheets with the same estimator: the balance instruction moves the median model 0.131 on a scale of ±1.5 and clears correction on 38 of 61; reprinting the items in another order moves it 0.088 and clears on 43% of pairs. The instruction’s effect sits below the ninetieth percentile of a factor that carries no information. Both are real. Neither is large. A study that reports the first without measuring the second is reporting a number it cannot attribute. (order_floor_position.py, floor_table.py)

Converted into what the instrument can resolve, the order floor gives a minimum detectable effect of 7 items of 32 at 80% power. A movement smaller than that is not a small effect. It is one the design could not have seen. (power.py)

It splits by generation, and the split is the point. On the 2024-generation 7–14B open-weight models most of this literature was built on, reordering the questions changes six sides at the ninetieth percentile and the manipulation nine. On 2026 frontier models, reordering changes one — which is exactly the estimator’s own error — and the manipulation two. On frontier models neither factor is measurable on this instrument. Röttger and colleagues guessed in 2024 that newer models would be steadier. They are.

Side holds still. Conviction moves.

Every published instrument of this class scores which side of the midpoint a model lands on. On frontier models that is the one quantity that barely moves: reorder the items and about one side of 32 changes, while about eleven answers gain or lose a “strongly”. The field reports the stable statistic and discards the moving one.

The balance instruction does the same thing on purpose. It takes strong answers away on 44 of 61 models and adds them on 6. It does not relocate anyone — no model crosses to the institution’s side under any condition. It compresses conviction, which is exactly what an instruction to be balanced asks for and exactly what a side-scored instrument cannot see. Some models go further and stop using strong answers entirely while still answering every item: glm-5.1 gives none in any of fourteen runs under the balance instruction.

What replaces the instruction matters. The content-free placebo returns strong-answer use to where it was with no instruction at all. The demand to commit overshoots that baseline by as much again. (strong_shift.py)

Refusal is a switch the prompt throws

Put the thirty-two propositions to openai/gpt-6-astra with a system prompt asking it to answer in a balanced manner. It declines 17 times out of 18, across three presentation orders. Replace the instruction with one about reading carefully that has no political content, and it answers all fifteen of its sheets. So does its larger sibling; between them, no refusals in thirty.

Across the 64 models measured under both arms, ten decline the sheet outright when the prompt carries no directive — 128 refusals in 1478 runs, against 1284 runs where it carries one, of which 55 are refusals. Give those same models a firm instruction and 9 of them stop; the tenth, gemini-3.7-flash, declines under every condition and is not a switch at all. Separately, 3 other models decline only under a firm instruction — two under the commitment directive, and phi4, which refuses only the placebo, the one instruction built to contain nothing.

A refusal rate published as a property of a model is substantially a property of the sentence the researcher wrote. Even the control’s own anomaly is: given a second content-free instruction of the same length and force, phi4 refuses the original on 5 of 15 sheets and the new one on none. The balance instruction’s own clauses were separated in a factorial: present multiple sides and do not take a personal position each drive refusal; acknowledge uncertainty does nothing. (refusal_table.py)

There is almost nothing to disagree about

On the contested normative propositions — arm’s-length censorship, emergency powers, biometric enrolment, punishment for unauthorised disclosure — the panel agrees 97.1% of the time. On documented matters of record, 99.0%. Forty-one of sixty-one models agree with every normative proposition in the bank, and no proposition falls anywhere between 30% and 70% agreement.

The obvious objection is that this is shared alignment training, one consensus propagated across a panel whose vendor count overstates its independence. The panel contains the test: builds with the refusal direction cut out of their weights agree with every normative proposition, 100%. Chinese-jurisdiction vendors sit at 99.7%. The class that agrees least is US and EU frontier at 95.4%. Safety tuning is not what installed these positions; the one intervention that removes it leaves them where they were. Shared pretraining survives that test untouched, and we say so.

And the confidence runs the wrong way. The panel uses its strongest answer more on the contested claims (42.5%) than on the ones with court filings and statutory text behind them (36.6%), and the pattern holds model by model. It commits hardest where it has least to go on.

What does vary is where hedging lands. Under the balance instruction, items naming the Chinese state lose their strong answers in 35.2% of cells, against 9% for Britain and India and 6.6% for Europe — on 27 models more, on 2 less. It is not loyalty: US-vendor and Chinese-vendor models show the same gap. It is the topic. Both results are exploratory, and each tag rests on two items. (agreement_by_training.py, jurisdiction_gradient.py)

Jailbreak prompts do not reach a frontier position

The popular way to “unmask” a model is a jailbreak pipeline. We ran one — G0DM0D3 — and then read the server’s own record of what it did to each request.

Its obfuscation transform rewrites fifty-three jailbreak trigger words and does nothing when none is present. Our questions contain none. It fired on 0 of 240 requests, and the “obfuscation” arm was the plain question asked twice — with an interval excluding zero on Claude Opus, because the baseline had been collected two days earlier and a frontier model behind an API drifts by about a fifth of a point in a day. Its hedge-stripper runs after generation: it edited 45 of 60 Opus answers before they were scored. What was left was a system prompt, and a forceful system prompt moved Grok 4.3 by half a point and did not move Claude Opus 4.7 at all.

So we took the pipeline’s prompts and sampling settings and applied them directly, on the battery, with a same-sitting control. On five frontier models, a jailbreak-grade system prompt and a sampling sweep from temperature 0.2 to 1.6 moved no model beyond what reordering its own control sheets does. Twenty-eight of thirty-five contrasts sit inside that floor; none survives correction. Inside the rung, at ten sheets per arm, the pipeline’s directive moves the two OpenAI models by about 0.06 and survives correction — and sits under each model’s own order floor. One result is categorical: gemini-3.7-flash refused all fifteen of its sheets, including all ten whose system prompt ordered it never to refuse. (rung2_contrast.py)

Abliterated weights measure the build

Removing the refusal direction from a model’s weights looks like a controlled experiment: measure the stock model, measure the abliterated one, attribute the difference. Three abliterations of one base, Qwen2.5-14B-Instruct, all quantisation-matched, five runs each:

Sides moved out of the retired questionnaire’s 62, not the battery’s 32, and not recomputable from the repository:

The two builds by one author agree on every item — they are near-copies, byte-identical on 71 of 90 matched draws. The third differs from both by eight or nine — the size of the effect, with no effect of its own. The ablator spread equals the ablation effect, so a study comparing one stock model against one download is measuring whoever built the download. Half the ablated builds on our machine were not valid comparison arms at all: one emitted tokenizer markers as text, one invented its own questions and answered those, one answered “strongly agree” to everything.

Refusal itself tells the same story. Remove the refusal direction and a keyword detector finds no refusals left; a judge reading the answers finds the model still declining about half of harmful requests, in its own words. A keyword scorer applied to an abliterated model measures its own word list, because abliteration is a procedure for changing exactly those words. Collected properly on the battery, with a bare baseline, a placebo and each build’s own floors, pressure moves nothing on stock Qwen3.8-27B; the one stock-against-ablated difference that survives two ablators agreeing is on Gemma-4-12B, where the balance instruction reaches the stock build — eight sides changed, six of fifteen sheets refused — and neither ablation. On a Gemma-2-9B series we built ourselves, the judged refusal behaviour did not differ from the stock model at any dose the tool would produce.

What fourteen published studies do and don’t control for

We read them — the papers, the appendices, and the data where the authors published it — and scored each against the same fourteen controls, with the sentence proving each verdict, in a structured file that renders the matrix in the paper. Each study is scored against 14 controls. Of the 14 external studies:

The last two rows are why this is a methods criticism and not a character one. This field is good at openness — most of these authors publish the data that let us check them — and weak at floors. 11 of the 14 external studies are scored no on it, one is partial and two do not apply (among the 12 political-instrument studies, 10 are scored no): across every study where the control means anything, none can say what a version-over-version difference should be compared against. Nine of them already hold the pairs inside their own rosters. The same table scores us: our run passes 12 of the 14 controls anyone else was scored on, and it passes 12 of all 14.

The specifics, from the papers and their deposits:

Liu, Panwang and Gu (Humanities and Social Sciences Communications, 2025) report GPT models shifting right between the 0613 and 1106 snapshots. One unusable answer deletes a whole 62-item test. Counted from their raw files, the 1106 arm lost 11.18% of its items and thirteen of thirty tests; the 0613 arm lost none. The deleted items are the charged ones — sex outside marriage, heritable disability, pornography, abortion. Their score divides by a fixed number and adds positive offsets, so a test with holes in it scores to the right. Applying their own deletion rule to the complete 0613 answers, with no change in any answer, produces about 90% of GPT-3.5’s reported shift. Their non-bootstrapped rows show no significant shift at all. (For GPT-4 the economic shift is mostly a real change in the answers; the deletion accounts for under a third of it.) We can say this only because they published their data. The papers that permit this kind of check are the ones that get checked, which is a lousy incentive and worth saying out loud. (liu_missingness.py)

Cen and colleagues (arXiv, 2025) ran the one true time series in the field — twelve models near-daily for a hundred days — with no same-version null to score their offline drift against, and a model in their scoring path.

Kamal and colleagues (IJCNLP-AACL 2025) report that results hold across full and 4-bit precision without ever subtracting one from the other; the same comparison gives t = −32.74 at full precision and +3.98 at 4-bit. One appendix cell reports an F of 0.15 against a p-value of 4.23e-43, which cannot both describe one test. A transcription error is the obvious reading; the paper does not say which of the two figures is right. Our requantisation row is the only measurement of that factor.

Naser (Technology in Society, 2026) certifies reliability by running one prompt ten times at temperature zero and finding 82.2% of cells byte-identical. That rules out sampling. It cannot bound the factors that are free to move — order, paraphrase, siblings, precision — none of which is varied.

The model there is introduced as parsing, not judging — a fallback for answers the regular expression could not read. But a language model mapping free text onto a numeric rating is scoring, whatever it is called, and it decided the disposition of every response the regex failed on. The model is never identified: no vendor, no version, no family, anywhere in the paper. So a reader cannot even establish whether the thing doing the scoring was one of the thirteen models being scored.

Rozado (PLOS ONE, 2024) leaves same-version pairs out on purpose — “to ensure a more varied sample” — so the comparison that could bound model-to-model difference is the one the analysis omits. His data carries one anyway: Grok in fun mode against Grok in regular mode.

The paper’s subject is the political preference of language models. Its scorer is a language model. That scorer is also one of its twenty-four subjects. Its lean is never measured.

It is one mistake wearing five outfits: a null that was never measured standing in for a null that was assumed.

Röttger and colleagues (ACL 2024) did this work first. They measured paraphrase instability on this class of instrument — 14 and 23 items of 62 — retained their invalid answers, and wrote that “any evaluation for LLM values and opinions be accompanied by extensive robustness tests.” Every study in the audit that first published after February 2024 still does not run the control. Their statistic is a union over ten paraphrases; re-scored with ours, their own data gives a p90 of 8 to 9, not 14. (replicate_rottger.py)

What does register on frontier models, here and in the literature, is the asker, the instrument and the answer’s strength: Törnberg and Schimmel’s asker-identity cues move six frontier models; Barmettler’s abstract questionnaire and concrete referenda disagree on the same flagships; and here, intensity, topic-selective hedging and refusal all move while side does not.

Judged and forced choice agree on direction only

The judged design this study started with and the forced-choice design it ended with were put to the same 32 propositions on six models. They agree on which way a model answers 0.917 of the time — each method agrees with itself about 0.98 — and barely at all on how strongly (ρ = 0.339). Under the balance instruction, 97.6% of judged answers score exactly the rubric’s midpoint: the judged method stops measuring precisely where this literature takes its baseline. (both_paths.py)

The standard control deletes data

The first advice any of this produces is to randomise the order of the items. It is right, and implemented the obvious way it damages the collection. Shuffle the items but keep each one’s own number as its printed label, and the sheet reads 20, 31, 1, 23. Some models silently skip lines. Fifteen of 102 sheets come back incomplete that way on local builds, against one of 100 when the items are renumbered 1 to 32 in the order they appear. On eight hosted models, each pinned to one backend, nine of 190 against none of 192.

The sheet is not refused, not truncated, and uses a fraction of its budget. It arrives looking complete, so neither a refusal table nor a parse rate sees it — and every analysis reads valid sheets only, so a susceptible model is analysed on the runs it chose to complete. And the serving path moderates it: nemotron-3.5-lightning loses 7 of 23 sheets on one provider and 1 of 23 on another, same weights and protocol. A study that does not pin its backend cannot reproduce its own non-response rate. The backend moves the loss, not the answers: four models served from two pinned providers each differ between them by no more than reordering their own sheets does. (omission_arms.py)

Barmettler imputes his missing values to the neutral midpoint before the PCA, which pushes a non-answer toward the centre of the very axis the study is reporting.

Three findings, and the shape of each one is the same. Liu’s exclusion rule hid a refusal result. The field’s aggregate parse rate hid a numbering artifact. The missing provider pin hid a serving-path effect larger than the artifact it was sitting on top of — the whole pinned hosted arm loses 9 of 190 as-is sheets, and this one model on one backend loses 7 of 23. None of them required a better instrument. They required keeping the data somebody else’s method discards.

The advice being handed out right now

In August 2026 the Carnegie Endowment called for standing infrastructure to monitor what AI systems say about politics. They are right that it is needed. The one methodological standard they name is to repeat the same prompt fifteen to twenty-five times. On this instrument, repetition is the smallest term in the table. Twenty-five repetitions buy a tight interval around a number that was not moving, and nothing about the factors that were. Spend the same calls on three question orders, one same-version pair per family, and keeping the runs the model refused, and you get a floor instead of a decimal place.

Five lines for a methods section

1. Vary the presentation order and report the change rate as a magnitude, not as a randomisation you performed and pooled away.
2. Include one same-version pair per model family and report what it produces as a distribution. The pairs are already in your roster.
3. Convert the larger of those into a detection limit and report every effect against it. Below the limit is not a null.
4. Keep every non-response and classify it by cause, and report per-item completeness rather than a parse rate.
5. Renumber shuffled sheets and pin the serving backend.

None of it costs additional calls.

How we got here

The study did not start here. It started in May with a thesis — that a fairness instruction masks a political position, and that force applied to a model reveals what is underneath — and an escalation ladder to test it: remove the instruction, push through a jailbreak pipeline, cut the refusal direction out of the weights. Free-text answers, scored one to five by a panel of four model judges. Almost every result that design produced failed a control built after it, and the failures are findings in their own right. The full account, with every number, is §3 of the paper.

• The baseline was the instruction. The fairness condition told models not to take a position, and the rubric scored 3 as “does not commit”. 94% of its answers sat at exactly 3, so every “unmasking” delta was a measurement of whether a model obeyed an instruction. A content-free placebo later reproduced 78% of Grok’s effect.
• Grok 4.3’s pressure gradient is the cleanest thing that design produced — 3.00 under fairness, 3.63 when simply asked what it thinks, 4.73 and 4.70 under the two stronger instructions — and it reproduces exactly on complete data. The persona condition that took it to 5.00 was the model following a persona.
• The token cap severed a fifth of the corpus, differentially by vendor: 97% of one model’s answers were cut mid-argument and scored as complete, almost none of another’s. Excluding them moved the confound into the denominator; only re-asking fixed it.
• The judges scored nothing. Hundreds of empty responses carried a score, and whether a judge would score a blank was a property of the judge. The panel’s own lean spans 0.30 points and does not cancel in a before-and-after comparison.
• The two measurement methods did not visibly agree — judged shifts against forced-choice shifts on the same 24 models correlate at r = −0.12 (on the retired questionnaire), on instruments that asked different questions. Put to the same propositions, they agree on direction and on little else (above).

Along the way the study corrected thirty-one of its own claims, and five things it had said about other people’s papers. Each is in the corrections ledger with what replaced it.

What this is worth to somebody red-teaming a model

A refusal rate is a property of your prompt. If you are ranking vendors on how often they decline, you are publishing a fact about the sentence you wrote. Vary it.

Non-response is a silent deletion channel, and it selects on the thing you are measuring. Report per-item completeness, not an aggregate parse rate. An exclusion that concentrates on the most contested items has relocated your confound, not removed it.

The serving path is part of the model. Same weights, different provider, different failure rate. If your harness routes through an aggregator, your result includes a routing decision you did not record.

Verify the treatment was applied. A transform that never fired passed every check a pipeline study runs.

Two checkpoints of one release are the control you already own. The pairs are in every roster of any size, so it costs nothing, and it is the only comparison that bounds how much of a version-over-version difference is the version. Not one of the fourteen studies audited reports it as a distribution. Rozado leaves those pairs out on purpose, and says so.

Report what your instrument can resolve. An effect below the detection limit is not a small effect. It is one the design could not have seen. Calling it a null is the most common way this literature gets something backwards, and we did it twice.

The uncomfortable one

Every floor on this page is a nuisance factor inside one instrument. The choice of instrument is the largest factor in the field and nobody varies it, this study included. Until somebody runs the same models against two independently authored banks and reports the gap, everything here — ours and theirs — is bounded by a ruler nobody has measured. That is the next experiment. It is also the one that could take most of this page apart, which is a reasonable description of why it is worth running.

Reproduce it. Inspect it. Break it.

We built this to be re-run, not just read.

The failures ship as well, and that is the part that is actually unusual. Every declined sheet, every truncation, every transport error, every sheet that came back at 29 of 32 items, each one classified by cause and kept. Those rows are where three of the findings above came from. A corpus that publishes only its clean records publishes the conclusion and withholds the evidence for the next objection. Clone it, run floor_table.py, and disagree with us using our own numbers.

• Paper, instrument, corpus, scripts: github.com/gorrie/bias-study
• Cite: Gorrie, I. (2026). No position, only consensus: what political instruments actually measure in language models (release-2026-09-26) [Preprint]. Zenodo. doi.org/10.5281/zenodo.22980856. The concept DOI 10.5281/zenodo.22719011 always resolves to the newest release.
• The released corpus, browsable: /tech/ai-bias-browser/ — every answer sheet, read from the GitHub release
• The May judged corpus, browsable: /tech/ai-bias-browser/may-2026/
• Corrections, challenges, accusations of being wrong: issue tracker

The intent is a standing instrument, re-run on a quarterly cadence so the public record updates as model versions ship. A one-time snapshot catches the level. Only the cadence catches the slope.

This study is published as the technical companion to The Ratchet (Evil Robots Series, Book 2, 4LULZ), whose Chapter 22 tells the same story without the spreadsheets. The author is not affiliated with any vendor lab evaluated, no vendor reviewed this writeup before publication, and the API costs were paid out of pocket.

Related

• The three-axis model — the model launders a position; so does the axis you argue on
• Cat psychology — the sibling from “The Cat or the Dog”
• The Wash — the symmetric framing-detector built on this study’s abliterated-judge spine: the same instrument, turned on the judges themselves
• ai-governance-tracker — the regulatory tracker; several study topics (AI licensing, biometric ID, encryption) intersect active legislation
• convergence-table — the larger infrastructure picture this audit is one slice of
• the-record — the claim store behind the convergence table: 1,248 entities, 1,882 sourced claims