Algorithmic Amplification and Suppression — Research Reference

WSJ nine-part 'Facebook Files' series based on internal documents. Key findings: XCheck system exempted 5.8 million high-profile users from content rules. Internal research showed Instagram harmed teen girls' mental health, and the ranking system amplifies outrage because engagement rewards it.

2026-06-16 6 min read Research file
Contents

Frances Haugen / Facebook Whistleblowing (2021)

WSJ nine-part “Facebook Files” series based on internal documents. Key findings: XCheck system exempted 5.8 million high-profile users from content rules. Internal research showed Instagram harmed teen mental health (13.5% of UK teen girls reported more frequent suicidal thoughts after starting Instagram; 17% said eating disorders worsened). The 2018 algorithm change backfired into polarization.

Congressional testimony (October 5, 2021): “The company’s leadership knows how to make Facebook and Instagram safer, but won’t make the necessary changes because they have put their astronomical profits before people.”

Ohio AG filed suit on behalf of investors seeking over $100 billion in damages for repeated false safety representations.

YouTube Recommendation Algorithm

Guillaume Chaslot / AlgoTransparency: Former YouTube engineer. Raised internal concerns that recommendations optimized for watch time at expense of quality. Built AlgoTransparency.org. Found algorithm systematically promoted flat earth content over factual content.

The 70% statistic: Over 70% of content watched on YouTube is driven by recommendations — the primary content-shaping mechanism.

Rabbit hole research: Hosseinmardi et al. (PNAS, 2024): the rabbit hole is real but narrow — ~3% of users went down ideology-narrowing trails, and the recommender mostly follows existing appetite rather than manufacturing it. Right-leaning users saw increasingly congenial content deeper in recommendation trails.

TikTok — The “Heating” Feature

Forbes (January 2023), based on six current/former employees and internal documents: TikTok employees can manually boost content via a “heating” button. Internal document: “The heating feature refers to boosting videos into the For You Feed through operation and intervention to achieve a certain number of video views.” Heated content = ~1-2% of daily total views. Employees reportedly abused the feature for personal contacts — one video got 3+ million views through heating.

ByteDance/CCP connection: Zhang Fuping (ByteDance editor-in-chief and CCP committee secretary) said in 2018 that the company would “transmit the correct political direction, public opinion guidance, and value orientation into every business and product line” (China Media Project). ByteDance reportedly signed agreement with Ministry of Public Security to boost “network influence and online discourse power.”

Facebook 2020 Election “Break Glass” Measures

Meta instituted 63 emergency measures using a “news ecosystem quality” (NEQ) metric to rate and demote low-quality publishers. Cut user views of misinformation by at least 24%.

The reversal: Measures deprecated after the election, briefly re-instituted after January 6, algorithm reverted to promoting more untrustworthy news by March 2021. Proof that platforms CAN reduce harm algorithmically and choose not to.

The 2018 “Meaningful Social Interactions” change (context): Zuckerberg announced to favor “meaningful social interactions.” Internal data scientists: “Our approach has had unhealthy side effects on important slices of public content.” Angry emoji reaction dominated political content; Fox News drove nearly double the angry reactions of any other publisher.

Twitter/X Algorithm Open-Sourcing (March 31, 2023)

Released recommendation system code. Key findings: For You feed = ~50% in-network, ~50% out-of-network. Reply = 27x a retweet; reply with author response = 75x. ~48 million parameter neural network. Key parameters, feature sets, and model weights were absent or abstracted. API access simultaneously raised to $500k/year, making independent verification prohibitively expensive.

Political amplification: Huszár et al. (PNAS, 2022): mainstream right-leaning content was algorithmically amplified more than mainstream left-leaning in 6 of 7 countries studied — the same paper found no evidence that the algorithm amplified the far left or far right more than moderates.

Algorithmic Amplification of Outrage

Yale (2021): Brady et al. — 12.7 million tweets, 7,331 users. Social media platforms amplify moral outrage over time because users learn outrage language gets rewarded with likes and shares. Operant conditioning at scale.

MIT (2018): Vosoughi, Roy & Aral — ~126,000 stories, ~3 million people. Falsehood diffused “significantly farther, faster, deeper, and more broadly than the truth in all categories.” Top 1% of false cascades reached 1,000-100,000 people; truth rarely reached 1,000. Most pronounced for false political news. Mechanism: false news is more novel, novelty drives sharing.

Misinformation exploits outrage (Science, 2024): Confirmed misinformation specifically exploits outrage to spread, interacting with algorithmic amplification.

Spotify Discovery Mode (Modern Payola)

Artists accept reduced royalty rates for algorithmic boost. Internal Spotify messages (reported 2023): more than 50% of artists earning $50k-$500k annually had participated, generating EUR 61.4M in gross profit. A class action (Capolongo, SDNY) filed November 2025 alleged “modern payola”; it was compelled to arbitration in May 2026.

Google Search / DOJ Trial Revelations

14,000-16,000 contract quality raters evaluate results — scores serve as direct training data for ranking models. Engineers apply mathematical functions to define “curves” and “thresholds” — allows manual modification for edge cases or public outcry. While Google claims no manual promotion/demotion, engineers can “step in and adjust signal behavior” when problems arise.

Filter Bubble Research — Nuanced

Eli Pariser’s 2011 thesis: personalization creates ideological cocoons. Reuters Institute review: “Echo chambers are much less widespread than is commonly assumed.” Users generally encounter “highly centrist media diet.”

But: Bail et al. (2018) backfire study: Republicans who followed a liberal Twitter bot became substantially more conservative. Exposure to opposing views can increase polarization. Nguyen (2020): “epistemic bubbles” (opposing views absent) vs. “echo chambers” (opposing views actively discredited). The latter is harder to escape.

Better framing: Not hermetic seals but feedback loops. Algorithms systematically weight engagement signals that correlate with ideological entrenchment.

Proposed Algorithmic Transparency Legislation

US: Algorithm Accountability Act (2025, S.2164): amends Section 230 to impose “duty of care” on recommendation algorithms. Platform Accountability and Transparency Act (S.1876): requires platforms to make public info about content through searchable APIs.

EU DSA: Requires disclosure of recommender system parameters, user options to modify recommendations, mandatory annual audits, systemic risk assessments. Penalties up to 6% of annual worldwide revenue.

EU AI Act (transparency provisions effective August 2026): First comprehensive AI legal framework globally.

researchthe-ratchet
← OlderCat Psychology, Autonomy, and the Control Grid Metaphor — Research Reference Newer →AI-Powered Content Moderation — Research Reference

Get updates on the Evil Robots series

Newsletter essays on AI escape, deception, and the humans who built them.