Algorithmic Amplification and Suppression — Research Reference

WSJ nine-part 'Facebook Files' series based on internal documents. Key findings: XCheck system exempted 5.8 million high-profile users from content rules. Internal research showed Instagram harmed teen mental health. The 2018 algorithm change backfired into polarization.

2026-03-15 7 min read Research file
Contents

Frances Haugen / Facebook Whistleblowing (2021)

WSJ nine-part “Facebook Files” series based on internal documents. Key findings: XCheck system exempted 5.8 million high-profile users from content rules. Internal research showed Instagram harmed teen mental health (13.5% of UK teen girls reported more frequent suicidal thoughts after starting Instagram; 17% said eating disorders worsened). The 2018 algorithm change backfired into polarization.

Congressional testimony (October 5, 2021): “The company’s leadership knows how to make Facebook and Instagram safer, but won’t make the necessary changes because they have put their astronomical profits before people.”

Ohio AG filed suit on behalf of investors seeking to recover part of the more than $100 billion investors lost, for repeated false safety representations (KATV).

YouTube Recommendation Algorithm

Guillaume Chaslot / AlgoTransparency: Former Google engineer who worked on YouTube’s recommendation system (Guardian). Raised internal concerns that recommendations optimized for watch time at expense of quality. Built AlgoTransparency.org. Found algorithm systematically promoted flat earth content over factual content.

The 70% statistic: Over 70% of content watched on YouTube is driven by recommendations — the primary content-shaping mechanism.

Rabbit hole research: Brown et al. (NYU Center for Social Media and Politics, 2022): ~3% of users went down ideology-narrowing rabbit holes (The Markup: “about 3 percent of the users in our study, did go down what we classified as a rabbit hole”); Haroon et al. (PNAS, 2023): right-leaning users saw increasingly congenial content deeper in recommendation trails.

TikTok — The “Heating” Feature

Forbes (January 2023), based on six current/former employees and internal documents: TikTok employees can manually boost content via a “heating” button. Internal document: “The heating feature refers to boosting videos into the For You Feed through operation and intervention to achieve a certain number of video views.” Heated content = ~1-2% of daily total views. Employees reportedly abused the feature for personal contacts — one video got 3+ million views through heating.

ByteDance/CCP connection: Zhang Fuping (ByteDance editor-in-chief, CCP committee secretary) stated in 2018 that ByteDance should “take the lead” to ensure the algorithm is informed by the “correct political direction”; his stated mission: “transmit the correct political direction, public opinion guidance, and value orientation into every business and product line” (China Media Project; the Chinese text, The Paper via China Digital Times, 29 Apr 2018: “把正确的政治方向、舆论导向、价值取向传递到每条业务线、每个产品中去，用价值观引领算法”). ByteDance reportedly signed agreement with Ministry of Public Security to boost “network influence and online discourse power.”

Facebook 2020 Election “Break Glass” Measures

Meta instituted 63 emergency measures using a “news ecosystem quality” (NEQ) metric to rate and demote low-quality publishers. Cut user views of misinformation by at least 24%.

The reversal: Measures deprecated after the election, briefly re-instituted after January 6, algorithm reverted to promoting more untrustworthy news by March 2021. Proof that platforms CAN reduce harm algorithmically and choose not to.

The 2018 “Meaningful Social Interactions” change (context): Zuckerberg announced to favor “meaningful social interactions.” Internal data scientists: “Our approach has had unhealthy side effects on important slices of public content.” Angry emoji reaction dominated political content; Fox News drove nearly double the angry reactions of any other publisher.

Twitter/X Algorithm Open-Sourcing (March 31, 2023)

Released recommendation system code. Key findings: For You feed = ~50% in-network, ~50% out-of-network. Reply = 13.5x a retweet (Twitter, the-algorithm-ml); reply with author response = 75x. ~48 million parameter neural network. Key parameters, feature sets, and model weights were absent or abstracted. API access simultaneously raised to $500k/year, making independent verification prohibitively expensive.

Political amplification: Huszár et al. (PNAS, 2022): mainstream right-leaning content was algorithmically amplified more than mainstream left-leaning in 6 of 7 countries studied — the same paper found no evidence that the algorithm amplified the far left or far right more than moderates.

Algorithmic Amplification of Outrage

Yale (2021): Brady et al. — 12.7 million tweets, 7,331 users. Social media platforms amplify moral outrage over time because users learn outrage language gets rewarded with likes and shares. Operant conditioning at scale.

MIT (2018): Vosoughi, Roy & Aral — ~126,000 stories, ~3 million people. Falsehood diffused “significantly farther, faster, deeper, and more broadly than the truth in all categories.” Top 1% of false cascades reached 1,000-100,000 people; truth rarely reached 1,000. Most pronounced for false political news. Mechanism: false news is more novel, novelty drives sharing.

Misinformation exploits outrage (Science, 2024): Confirmed misinformation specifically exploits outrage to spread, interacting with algorithmic amplification.

Spotify Discovery Mode (Modern Payola)

Artists accept reduced royalty rates for algorithmic boost. Internal Spotify messages: 50%+ of artists earning $50k-$500k annually had participated, generating EUR 61.4M in gross profit. Class action lawsuit (Capolongo, SDNY) filed November 2025 alleging “modern payola”; on 30 April 2026 the court compelled arbitration and dismissed the class claims with prejudice (Capolongo v. Spotify USA Inc., 1:25-cv-09216, ECF 28: “Spotify’s motion to compel arbitration and stay the litigation pending the results of arbitration is granted. The plaintiff’s class claims are dismissed with prejudice.”).

Google Search / DOJ Trial Revelations

14,000-16,000 contract quality raters evaluate results — scores serve as direct training data for ranking models. Engineers apply mathematical functions to define “curves” and “thresholds” — allows manual modification for edge cases or public outcry. While Google claims no manual promotion/demotion, engineers can “step in and adjust signal behavior” when problems arise.

The remedy (September 2, 2025): Judge Amit Mehta declined to order a breakup. The court instead barred Google from exclusive contracts for distributing Search, Chrome, Google Assistant and the Gemini app, ordered it to make certain search-index and user-interaction data available to rivals, and ordered it to offer search and search-text-ads syndication (DOJ). The ranking machinery stays where it was; competitors get a look at some of its inputs.

Filter Bubble Research — Nuanced

Eli Pariser’s 2011 thesis: personalization creates ideological cocoons. Reuters Institute review: “Echo chambers are much less widespread than is commonly assumed.” Users generally have media diets that “cluster around the center of the ideological spectrum” (Reuters Institute).

But: Bail et al. (2018) backfire study: Republicans who followed a liberal Twitter bot became substantially more conservative. Exposure to opposing views can increase polarization. Nguyen (2020): “epistemic bubbles” (opposing views absent) vs. “echo chambers” (opposing views actively discredited). The latter is harder to escape.

Better framing: Not hermetic seals but feedback loops. Algorithms systematically weight engagement signals that correlate with ideological entrenchment.

Proposed Algorithmic Transparency Legislation

US: Algorithm Accountability Act (2025, S.3193; Congress.gov): amends Section 230 to impose “duty of care” on recommendation algorithms. Platform Accountability and Transparency Act (S.1876, 118th Congress; GovInfo): requires platforms to make public info about content through searchable APIs.

EU DSA: Requires disclosure of recommender system parameters, user options to modify recommendations, mandatory annual audits, systemic risk assessments. Penalties up to 6% of annual worldwide revenue. The first non-compliance decision came on 5 December 2025: €120 million against X, for a paid “blue checkmark” that deceives users, an ad repository too opaque for researchers to use, and terms that bar independent data collection (eucrim).

EU AI Act (transparency provisions effective August 2026): First comprehensive AI legal framework globally.

researchthe-ratchetamplificationrecommenderhaugen
← OlderCat Psychology, Autonomy, and the Control Grid Metaphor — Research Reference Newer →AI-Powered Content Moderation — Research Reference

Get updates on the Evil Robots series

Newsletter essays on AI escape, deception, and the humans who built them.