2025-reranking-partisan-animosity-science
The paper reports a preregistered field experiment that used a browser extension and LLM‑based reranking to manipulate exposure to antidemocratic attitudes and partisan animosity (AAPA) content in users' X (Twitter) feeds. Two parallel experiments were run in July–August 2024 with 1,256 U.S. partisan participants: a Reduced Exposure condition that down‑ranked AAPA posts and an Increased Exposure condition that up‑ranked them. Participants were randomly assigned to treatment or control within each experiment. The study measured (i) affective polarization and a suite of eight political attitude items via in‑feed and post‑experiment surveys, (ii) emotional responses (anger, sadness, excitement, calm), (iii) engagement metrics (sessions, time spent, views, likes, reposts, and corresponding rates), and (iv) qualitative feedback on the extension. Demographic characteristics of the sample are detailed (education, income, race, SES ladder). Statistical analyses included linear, quasi‑Poisson, and Beta regressions, with False Discovery Rate (FDR) adjustments for multiple comparisons across primary, secondary, and tertiary outcomes, as well as heterogeneous treatment effect tests. Key findings: reducing AAPA exposure modestly warmed out‑party feeling (+2.1 points in‑feed, +3.2 post‑experiment) and lowered negative emotions; increasing exposure produced symmetric cooling (‑2.5 points) and raised negative emotions, though many secondary attitude effects were not significant after FDR correction. Engagement volume fell in the reduced condition (≈‑170 views, ‑24 likes) while like and repost rates rose slightly; the increased condition showed no significant engagement changes. Heterogeneity analyses found no moderator reaching significance after correction. Reweighting the sample to match the X user population (by party, education, race) yielded Population Average Treatment Effects (PATE) that largely mirrored Sample Average Treatment Effects (SATE), with some loss of precision. Qualitative open‑ended responses indicated that 74% of participants did not notice any impact; machine‑learning classification could not predict condition above chance. No decay of treatment effects was observed across the seven‑day intervention, and survey‑fatigue checks showed stable response variance. Major political events during the study (e.g., Trump assassination attempt, RNC, Biden withdrawal) were controlled for and did not drive the main effects. The paper concludes that platform‑independent LLM‑based reranking can modestly shift affective polarization without overt user awareness, but effects are limited in magnitude and short‑term.
Topics
Demographic Profile of the Post‑Experiment Survey Sample
Provides detailed tables of the 1,256 U.S. partisan participants, covering education levels, household income brackets, racial/ethnic composition, and self‑reported socioeconomic ladder positions. Includes party affiliation breakdown and notes on sample representativeness for the field experiment.