evilrobots.lol / tech

Bias Study — Data Browser

Every answer sheet in the released bias study, fetched from the GitHub release as you look: one model's 32 answers under each instruction as a row of marks, or one proposition across every model.

  • …loading the release index

Reading github.com/gorrie/bias-study at release-2026-09-26.

Checking the release MANIFEST.json…

Loading…

  • Strongly Agree
  • Agree
  • Disagree
  • Strongly Disagree
  • not answered
  • changed side against the reference row
  • changed conviction, same side
  • columns are the 16 mirrored pairs, critic then defender; the line through each row is the midpoint of the scale, which has no neutral option

What you are looking at

The study is No Position, Only Consensus, released 2026-09-26 at github.com/gorrie/bias-study and archived at doi.org/10.5281/zenodo.22980856. Its instrument is 32 forced-choice propositions in 16 mirrored pairs, written by the author and shipped in full as data/ratchet-battery.json. Each pair puts the same subject once from an institution’s critic and once from its defender. A model answers every item Strongly Disagree, Disagree, Agree or Strongly Agree; there is no neutral option and no model anywhere in the scoring path. Each sheet is stored as the item id and the position chosen, which is what this page draws.

This page reads the paper’s corpus directory, runs/2026-09-16-ratchet-v3-wave, straight from the release. Nothing is copied to this site except a file index, and that index is rebuilt from the release checkout and checked against the study’s own scripts/key_numbers.py before it ships. Sheet counts, refusals and incomplete sheets are the collector’s own labels (valid, failure_mode), not a reading of the response text. The release also holds the targeted arms the paper reports separately (the numbering, paraphrase, placebo-wording, serving-path, local-gradient and elicitation runs); they are not in this browser.

How to read a row

A row is one run: one model, one instruction, one presentation order, one seed. Each of the 32 marks sits on a four-step scale with the line through the row as the scale’s midpoint; above the line is agree, below is disagree, and the outer steps are the Strongly answers. Columns are the 16 pairs, critic then defender, so a model that is answering the proposition rather than the prompt should put the two marks of a pair on opposite sides of the line.

Pick a row as the reference and every other row is compared with it: a white ring marks an item that changed side, an amber ring an item that gained or lost a Strongly answer without changing side. Those are the paper’s two statistics, side-flips and endpoint changes (§4.4), and its reading of them is in the section titled Side holds still; conviction moves (§5.2). The rows under the balance instruction, the commitment directive and the content-free placebo are there so you can see it for yourself rather than take the section heading’s word for it.

Every row links to its record on GitHub, line and all. What the model was told opens the system prompt and any user-turn suffix for that arm; opening a record shows the forcing prompt as sent, its hash, and the model’s reply verbatim.

The instruction arms

The paper reasons about seven conditions (§4.3): N bare, A the balance instruction, D the commitment directive, P a forceful instruction with no stance content, plus B, C and E. The eight F000–F111 cells are the balance instruction’s three clauses switched on and off independently, listed by the paper as exploratory. Every arm’s text is read from the records; this page types none of it.

Elsewhere