Within AI Virality
Why small accounts can make big fakes
AI tools let small accounts create highly shareable posts that can outrun follower-count expectations.
On this page
- How AI lowers production barriers
- Why follower count no longer predicts reach
- What readers should check before amplifying
Page outline Jump by section
Introduction
One of the most important changes in the age of AI-generated misinformation is that a large audience is no longer a prerequisite for creating a viral falsehood. On X, misleading AI-generated posts are increasingly emerging from relatively small accounts yet still achieving outsized reach. Research analysing more than 91,000 misleading posts identified through Community Notes found that AI-generated misinformation was more likely than conventional misinformation to originate from smaller accounts while also being significantly more likely to go viral. [arXiv]arxiv.orgCharacterizing AI-Generated Misinformation on Social Media15 May 2025 — AI-generated misinformation is more often centered on entert…
This shift matters because many people still use follower count as a shortcut for credibility or influence. Historically, producing convincing hoaxes often required specialist skills, organised networks, or access to large audiences. Generative AI has weakened those constraints. A user with a modest following can now create content that looks professionally edited, emotionally engaging, and highly shareable, increasing the chance that platform algorithms and ordinary users will do the distribution work for them. [arXiv]arxiv.orgCharacterizing AI-Generated Misinformation on Social Media15 May 2025 — AI-generated misinformation is more often centered on entert…
How AI lowers production barriers
Before modern generative AI tools, creating a convincing fake image, fabricated screenshot, edited video, or realistic quote graphic usually demanded technical knowledge and significant effort. Today, many of those tasks can be completed in minutes using consumer-facing AI systems. [UF Journalism and Communications]2024.jou.ufl.eduUF Journalism and CommunicationsAI and MisinformationAI tools make it easy for anyone to create fake images and news that are hard to dis…
For small accounts, this changes the economics of attention. The challenge is no longer producing content that looks polished. The challenge is finding content that triggers sharing behaviour. AI helps by making it easier to generate:
- Photorealistic but fictional images.
- Fake screenshots that resemble genuine posts or news reports.
- Short video clips with professional-looking visual effects.
- Emotionally optimised captions and headlines.
- Endless variations of the same theme for testing audience reactions.
The result is that individual users can experiment at a scale that previously resembled the output of media teams or coordinated influence operations. A small account can publish dozens of highly polished pieces of content, increasing the odds that one will attract widespread engagement. [UF Journalism and Communications]2024.jou.ufl.eduUF Journalism and CommunicationsAI and MisinformationAI tools make it easy for anyone to create fake images and news that are hard to dis…
Why follower count no longer predicts reach
The traditional social-media model assumed a reasonably strong relationship between audience size and visibility. Large accounts generally reached more people because they started with more followers. That relationship has weakened as recommendation systems increasingly prioritise engagement signals over direct follower relationships.
Research on information diffusion has repeatedly shown that follower numbers alone do not reliably predict virality. Content characteristics, network effects, reposting patterns, and platform recommendation systems often matter more than the size of the original account. [PMC]pmc.ncbi.nlm.nih.govFollowers do not dictate the virality of news outlets on social…by E Sangiorgio · 2024 · Cited by 18 — In terms of information spre…
AI-generated misinformation appears particularly suited to this environment. The large-scale study of Community Notes data found a striking combination: AI-generated misleading posts tended to come from smaller accounts yet achieved higher viral performance than comparable non-AI misinformation. [arXiv]arxiv.orgCharacterizing AI-Generated Misinformation on Social Media15 May 2025 — AI-generated misinformation is more often centered on entert…
Several factors help explain this pattern:
Novelty attracts engagement. AI-generated images and videos often contain unusual or surprising visuals that encourage people to react and repost before verifying authenticity. [Misinformation Review]misinforeview.hks.harvard.eduMisinformation ReviewThe spread of synthetic media on Xby G Corsi · Cited by 45 — While non-political content and images have a higher pr…
Entertainment spreads faster than argument. Researchers found AI-generated misinformation is disproportionately entertainment-oriented. Humorous or visually striking posts can travel widely even when users are unsure whether they are true. [arXiv]arxiv.orgCharacterizing AI-Generated Misinformation on Social Media15 May 2025 — AI-generated misinformation is more often centered on entert…
Algorithms reward reactions, not reputation. Recommendation systems typically observe engagement signals such as likes, replies, reposts, viewing time, and interaction velocity. A post that performs strongly can escape the limits of its creator’s follower base. [PMC]pmc.ncbi.nlm.nih.govFollowers do not dictate the virality of news outlets on social…by E Sangiorgio · 2024 · Cited by 18 — In terms of information spre…
Screenshots detach content from origins. Once a post is captured and reshared as an image, many viewers never see the original account. The credibility assessment that might have occurred if users noticed a tiny account can disappear entirely.
The rise of the “viral outsider”
Historically, misinformation researchers often focused on celebrities, politicians, media outlets, or large coordinated networks. AI introduces a different problem: the viral outsider.
A small account may have little influence most of the time yet occasionally produce a post that reaches millions. In this model, influence becomes episodic rather than permanent. The account does not need a loyal audience. It only needs one piece of content that aligns with platform incentives.
The Harvard-based study of synthetic media on X found that non-political synthetic content, especially visually engaging images, often had a high probability of reaching viral levels. Many of these posts were humorous, satirical, or entertainment-focused rather than traditional political propaganda. [Misinformation Review]misinforeview.hks.harvard.eduMisinformation ReviewThe spread of synthetic media on Xby G Corsi · Cited by 45 — While non-political content and images have a higher pr…
This makes detection harder for ordinary users. People are accustomed to questioning major political claims from prominent figures. They may be less sceptical of an amusing image or a surprising visual shared by an unknown account because it feels less consequential. Yet the same mechanisms that spread harmless entertainment can also spread misleading information.
When virality outruns correction
The small-account problem becomes more significant when combined with the speed of social sharing.
Even when misleading content is eventually challenged, corrective systems often operate more slowly than initial distribution. Analyses of Community Notes have repeatedly highlighted delays, limited coverage, and the fact that many misleading posts never receive visible corrections. [LSE Blogs]blogs.lse.ac.ukLSE Blogs Do Community Notes work?LSE Impact14 Jan 2025 —… Community Notes mostly fails to combat misinformation”…. Pingback: Désinformation en 2025: Comment lutter…
For a small account, this means the critical window is often the first few hours. If a synthetic image or fabricated screenshot gains momentum quickly enough, the account’s lack of followers becomes largely irrelevant. By the time contextual information appears, screenshots, reposts, and copies may already be circulating independently. [LSE Blogs]blogs.lse.ac.ukLSE Blogs Do Community Notes work?LSE Impact14 Jan 2025 —… Community Notes mostly fails to combat misinformation”…. Pingback: Désinformation en 2025: Comment lutter…
The issue is not simply whether corrections exist. It is whether corrections can travel as fast as the original content. In many cases, they do not. [Nature]nature.comCommunity-based fact-checking reduces the spread of…by Y Chuai · 2026 · Cited by 2 — In this work, we perform a large-scale quas…
What readers should check before amplifying
The emergence of small-account virality means that traditional credibility shortcuts are less reliable than they once were. A post can be widely shared without coming from a widely trusted source.
Before reposting or quoting a dramatic image, video, or screenshot, consider several questions:
- Who originally posted it? Trace the content back to its earliest visible source rather than judging it from a repost.
- Does the account have a history? Newly created or low-activity accounts deserve additional scrutiny.
- Is the content unusually polished for the source? Generative tools make professional-looking material easy to produce.
- Can the claim be verified elsewhere? Look for independent reporting or corroborating evidence.
- Is the post optimised for surprise, amusement, or outrage? Those emotional triggers are often central to viral spread.
- Has contextual information been added later? Community Notes, fact-checks, or corrections may appear after initial publication.
The key lesson is that audience size is no longer a dependable measure of potential impact. In the AI era, a small account can create a piece of misinformation that reaches millions not because the account itself is influential, but because the content is designed to travel. Critical thinking therefore requires paying more attention to evidence and provenance than to follower counts or apparent popularity.
Amazon book picks
Further Reading
Books and field guides related to Why small accounts can make big fakes. Use these as the next step if you want deeper reading beyond the article.
The Chaos Machine
Explains how platform dynamics can amplify content from unexpected sources, including small accounts.
Calling Bullshit
Provides practical tools for evaluating suspicious viral claims and media content.
Network Propaganda
First published 2018. Subjects: Politics & government, Presidents, united states, election, 2016, Communication in politics, Political ca...
eBay marketplace picks
Marketplace Samples
Example marketplace items related to this page. Use the search link to explore similar finds on eBay.
Endnotes
-
Source: arxiv.org
Link: https://arxiv.org/html/2505.10266v1Source snippet
Characterizing AI-Generated Misinformation on Social Media15 May 2025 — AI-generated misinformation is more often centered on entert...
Published: May 2025
-
Source: arxiv.org
Title: arXiv Characterizing AI-Generated Misinformation on Social Media
Link: https://arxiv.org/abs/2505.10266 -
Source: misinforeview.hks.harvard.edu
Link: https://misinforeview.hks.harvard.edu/wp-content/uploads/2024/06/corsi_synthetic_media_20240603.pdfSource snippet
Misinformation ReviewThe spread of synthetic media on Xby G Corsi · Cited by 45 — While non-political content and images have a higher pr...
-
Source: pmc.ncbi.nlm.nih.gov
Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC11235336/Source snippet
Followers do not dictate the virality of news outlets on social...by E Sangiorgio · 2024 · Cited by 18 — In terms of information spre...
-
Source: arxiv.org
Title: arXiv Timeliness, Consensus, and Composition of the Crowd: Community Notes on X
Link: https://arxiv.org/abs/2510.12559 -
Source: nature.com
Link: https://www.nature.com/articles/s41467-026-72597-0Source snippet
Community-based fact-checking reduces the spread of...by Y Chuai · 2026 · Cited by 2 — In this work, we perform a large-scale quas...
-
Source: arxiv.org
Link: https://arxiv.org/pdf/2505.10266Source snippet
Characterizing AI-Generated Misinformation on Social Mediaby C Drolsbach · 2025 · Cited by 16 — In this study, we conduct a large-scale e...
-
Source: ui.adsabs.harvard.edu
Link: https://ui.adsabs.harvard.edu/abs/arXiv%3A2505.10266Source snippet
AI-Generated Misinformation on Social MediaIn this study, we conduct a large-scale empirical analysis of AI-generated misinformation on t...
-
Source: misinforeview.hks.harvard.edu
Title: the spread of synthetic media on x
Link: https://misinforeview.hks.harvard.edu/article/the-spread-of-synthetic-media-on-x/Source snippet
spread of synthetic media on X3 Jun 2024 — This study examines the prevalence and characteristics of synthetic media on social media plat...
-
Source: 2024.jou.ufl.edu
Link: https://2024.jou.ufl.edu/page/ai-and-misinformationSource snippet
UF Journalism and CommunicationsAI and MisinformationAI tools make it easy for anyone to create fake images and news that are hard to dis...
-
Source: blogs.lse.ac.uk
Title: LSE Blogs Do Community Notes work?
Link: https://blogs.lse.ac.uk/impactofsocialsciences/2025/01/14/do-community-notes-work/Source snippet
LSE Impact14 Jan 2025 —... Community Notes mostly fails to combat misinformation”.... Pingback: Désinformation en 2025: Comment lutter...
-
Source: pmc.ncbi.nlm.nih.gov
Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC12924558/Source snippet
It often adopts a positive or...Read more...
-
Source: Wikipedia
Title: Community Notes
Link: https://en.wikipedia.org/wiki/Community_NotesSource snippet
Community NotesInitially shown to U.S. users only, notes were popularized in March 2022 over misinformation in the Russian invasion of...
Published: March 2022
Additional References
-
Source: ibm.com
Link: https://www.ibm.com/think/insights/ai-misinformationSource snippet
AI MisinformationAI-generated content can be rife with errors, from fake news headlines and terrible legal advice to pizza recipes featur...
-
Source: semanticscholar.org
Link: https://www.semanticscholar.org/paper/Characterizing-AI-Generated-Misinformation-on-Media-Drolsbach-Pr%C3%B6llochs/7e175dd6251551091ebd8023064351e37c12c621Source snippet
Characterizing AI-Generated Misinformation on Social MediaLarge-scale empirical analysis of AI-generated misinformation on the social med...
-
Source: cetas.turing.ac.uk
Link: https://cetas.turing.ac.uk/news/everything-could-go-wrong-xs-new-ai-written-community-notesSource snippet
that could go wrong with X's new AI-written...AI chatbots often struggle with nuance and context but are good at confidently providing a...
-
Source: theguardian.com
Title: fears ai factcheckers on x could increase promotion of conspiracy theories
Link: https://www.theguardian.com/technology/2025/jul/02/fears-ai-factcheckers-on-x-could-increase-promotion-of-conspiracy-theoriesSource snippet
Fears AI factcheckers on X could increase promotion of...2 Jul 2025 — Former UK minister says platform, which will use AI to draft commu...
-
Source: arstechnica.com
Title: everything that could go wrong with xs new ai written community notes
Link: https://arstechnica.com/tech-policy/2025/07/everything-that-could-go-wrong-with-xs-new-ai-written-community-notes/Source snippet
Everything that could go wrong with X's new AI-written...2 Jul 2025 — The platform plans to allow AI to write community notes, and that...
-
Source: pnas.org
Link: https://www.pnas.org/doi/10.1073/pnas.2503413122Source snippet
Community notes reduce engagement with and diffusion of...by I Slaughter · 2025 · Cited by 37 — We find that notes significantly reduce...
-
Source: news.yale.edu
Title: flagging misinformation social media reduces engagement study finds
Link: https://news.yale.edu/2025/09/25/flagging-misinformation-social-media-reduces-engagement-study-findsSource snippet
misinformation on social media reduces... - YaleNewsSep 25, 2025 — Community Notes enables X users to propose and vet fact-checking note...
-
Source: osf.io
Link: https://osf.io/preprints/osf/3a4feSource snippet
ced the spread of misleading posts by, on average, 61.4%.Read more...
-
Source: ddia.org
Title: a deep dive into xs community notes report
Link: https://ddia.org/en/a-deep-dive-into-xs-community-notes-reportSource snippet
A Deep Dive into X's Community NotesJul 9, 2025 — As AI-generated videos and [deepfakes]({{ 'deepfakes/' | relative_url }})... Community brings together leading voices to ex...
-
Source: osf.io
Title: Authors. Yun Wang. Author Assertions. Conflict of Interest.Read more
Link: https://osf.io/preprints/socarxiv/8bgu6Source snippet
A Systematic Review: The Spread of AI-Generated...by Y Wang · Cited by 1 — A Systematic Review: The Spread of AI-Generated Misinformatio...
Topic Tree