#Model-Drift

What a political instrument measures in a language model when nothing political has changed. 32 author-written propositions, 65 models, 3,897 runs, no language model in the scoring path: the deliberate manipulation is the same size as reprinting the questions in a different order, refusal is a switch the prompt throws, and the models barely disagree. An audit of fourteen published studies finds the control that would have shown it missing from all of them. Data, scripts and paper at github.com/gorrie/bias-study.