ChatGPT Exposure Polarizes Political Slant While Reducing Hostility

Political polarization is often treated as a package: stronger partisan positions arrive with more hostility toward opponents. Wang and colleagues report a different pattern after ChatGPT's public release in Reddit's largest political forum. Liberal-leaning authors shifted further liberal and conservative-leaning authors further conservative, while toxicity and hostility declined. The paper's central contribution is to separate ideological polarization from affective polarization rather than treating civility as a proxy for political moderation.

Core Contribution

The study uses ChatGPT's November 2022 release as a temporal shock and estimates difference-in-differences models across authors grouped by their pre-period political slant. Political slant is derived from the average partisan orientation of an author's comments; authors below zero are liberal-leaning and those above zero conservative-leaning. The analysis then asks whether the post-release change differs by group, with author fixed effects absorbing time-invariant differences among participants.

The authors argue that the ideological shift is concentrated among authors already at the partisan ends, rather than being driven by neutral or centrist users. Their proposed mechanism is algorithmic sycophancy: LLM-assisted replies tend to echo the viewpoint of the original post. That account matters because it distinguishes reinforcement in conversational replies from a simpler story in which users deploy models to write independently more extreme or more persuasive political arguments.

Technical Approach

The dataset contains 6,882,091 posts and comments, of which 5,604,844 comments come from liberal- or conservative-leaning authors. The main design models political slant as a function of a post-November indicator, partisan-group treatment status, their interaction, and author fixed effects. The interaction estimates whether an ideological group changed differently after the release.

Probable LLM assistance is measured indirectly, not observed. The paper combines comment length, passive-auxiliary-verb counts from spaCy, GPT-2 perplexity, and an OpenAI pretrained human-detection score. Lower perplexity and lower human probability are interpreted as signals of LLM-assisted writing. For exposition, comments below the 30th percentile of human probability are classified as LLM-assisted; the authors state that their baseline analyses retain the continuous score and report similar qualitative findings at 10th- and 50th-percentile cutoffs.

Civility is measured two ways: Detoxify's overall toxicity score and a word2vec-based hostility score defined by the cosine similarity between a comment vector and the vector for “hate.” The paper also scores anger, fear, surprise, and sadness. This makes the affective result more specific than a single moderation-model output.

Results and Analysis

At the author-day level, the post-release interaction is -0.0038 for liberal-leaning authors and 0.0033 for conservative-leaning authors, both reported at $p < 0.01$. The corresponding neutral-author coefficient is -0.0004. At the author-month level, the partisan estimates are -0.006 and 0.005, versus -0.0004 for neutral authors. The same directional pattern survives an aggregation that gives authors, rather than prolific commenters, equal weight.

The appendix indicates that the movement is most pronounced among ideologically extreme users: author-level slant decreases by 0.015 for extreme liberal authors and increases by 0.011 for extreme conservative authors. Regular and centrist groups show substantially smaller changes. That distribution fits a reinforcement account better than a broad forum-wide drift, although it does not identify which specific comments were produced with model help.

The paper also moves placebo treatment dates one year and six months earlier, and substitutes the 2022 midterm-election date. Those falsification tests do not reproduce the main post-ChatGPT partisan divergence; the authors use this to argue against a generic pretrend or an election-only explanation. The evidence is persuasive for an association around the release window, but the causal interpretation still rests on the comparability of partisan groups and on proxy-based detection of LLM assistance.

Limits in Practice

The findings concern one large Reddit political forum during the first three months after release, when generated text may have had more detectable linguistic signatures than it does now. The 30th-percentile classifier is explicitly probabilistic, and neither perplexity nor a detector can determine authorship perfectly. Finally, reduced hostility does not show that participants became less politically divided; the paper's more defensible reading is that civil tone and ideological distance can move in opposite directions.