Within Community Notes
Can human notes keep up with AI fakes?
AI tools can create plausible false images and claims faster than human note systems can reliably review them.
On this page
- Why generative posts strain human review
- What AI written note drafts might change
- Where human judgment remains necessary
Page outline Jump by section
Introduction
Generative AI has changed the economics of misinformation. A convincing fake image, fabricated quote, synthetic voice clip or polished false narrative can now be created in seconds and distributed to millions of people at negligible cost. Community Notes and similar public-correction systems were designed for a world in which misleading content was produced mainly by humans. The central challenge today is scale: AI can generate misleading content faster than human contributors can review, investigate and annotate it. As a result, even effective note systems face a growing race between creation and correction. Research on AI-generated misinformation identified through Community Notes shows that such content can achieve substantial virality despite often originating from relatively small accounts. [arXiv]arxiv.orgarXiv Characterizing AI-Generated Misinformation on Social MediaCharacterizing AI-Generated Misinformation on Social MediaMay 15, 2025…
This does not mean public notes are ineffective. Rather, it highlights a practical implementation problem. A correction system built around human judgment must cope with a flood of increasingly realistic and increasingly abundant synthetic content. The question is not simply whether notes can identify falsehoods, but whether they can do so quickly and consistently enough to matter.
Can human notes keep up with AI fakes?
The most important shift introduced by generative AI is volume. Previous misinformation campaigns often required specialised skills, coordination or significant labour. Modern image generators, video generators and large language models allow individuals to create many variations of misleading content in minutes. Reviews of generative AI and misinformation repeatedly identify increased production speed and lower creation costs as major concerns. [Springer]link.springer.comGenerative AI and misinformation: a scoping review of the role…by S Park · 2025 · Cited by 48 — Analyzing 24 empirical studies…
For Community Notes-style systems, every misleading post creates a potential workload:
- Someone must notice the content.
- A contributor must investigate it.
- Evidence must be gathered and cited.
- A note must be written.
- Other contributors must review and rate the note.
- The system must reach a threshold for public display.
That process can work well for individual claims, but generative AI increases the number of claims entering the pipeline. Even when contributors are highly motivated, review capacity grows much more slowly than content generation capacity. Researchers analysing AI-generated misinformation on X explicitly note that crowd-based identification methods help address scalability limits faced by traditional human review systems, yet the underlying challenge remains that misleading content can appear at enormous volume. [arXiv]arxiv.orgCharacterizing AI-Generated Misinformation on Social Media15 May 2025 — Our work: In this study, we characterize AI-generated misinf…
The timing problem is equally important. Studies of Community Notes have found meaningful delays between the appearance of a misleading post and the appearance of a publicly visible correction. In one analysis of health-related notes, the median delay before a note achieved helpful status was more than 17 hours. [arXiv]arxiv.orgBeyond the Crowd: LLM-Augmented Community Notes for Governing Health MisinformationOctober 13, 2025…
For many viral posts, the largest audience arrives long before that.
Why synthetic media is particularly difficult
Not all misinformation creates the same burden.
A false textual claim can often be checked against documents, statistics or previous reporting. AI-generated images, videos and audio introduce additional challenges because reviewers must first determine whether the media itself is authentic.
Recent research suggests that realistic AI-generated imagery can increase belief in false headlines and make corrections harder to remember. [Misinformation Review]misinforeview.hks.harvard.eduMisinformation ReviewPeople are more susceptible to misinformation with…November 11, 2025 — by S Guo · 2025 · Cited by 2 — In a pre-re… Human observers also struggle to distinguish increasingly sophisticated synthetic images from genuine ones. Studies on AI-generated faces, for example, show that even highly skilled recognisers can perform poorly without specialised training. [Live Science]livescience.comLive Science AI is getting better and better at generating facesThese faces often appear more realistic than actual human faces, a phenomenon termed "hyperrealism." In a series of experiments, super re…
This creates a double burden for note contributors:
- They must verify the factual claim.
- They may also need to verify the authenticity of the media used to support it.
As image and video generation improves, the second task becomes increasingly demanding.
Why generative posts strain human review
The scale problem is not simply about the number of posts. Generative AI changes the characteristics of misinformation itself.
Faster adaptation and variation
Traditional misinformation often spreads through repeated reposting of the same content. AI systems can instantly create multiple versions of the same narrative, changing wording, imagery, tone and framing.
A correction written for one version may not automatically apply to dozens of related variants. Human reviewers therefore face a moving target rather than a single false claim.
Lower barriers for small accounts
Research examining more than 91,000 misleading posts identified through Community Notes found that AI-generated misinformation frequently originates from smaller accounts yet still achieves significant viral reach. [arXiv]arxiv.orgarXiv Characterizing AI-Generated Misinformation on Social MediaCharacterizing AI-Generated Misinformation on Social MediaMay 15, 2025…
Historically, moderation systems could focus attention on a relatively limited set of influential accounts. AI-generated content allows many smaller actors to produce professional-looking misinformation that can spread widely if it aligns with audience interests or current events.
More persuasive presentation
Generative AI does not merely produce false statements. It can create convincing supporting material:
- fabricated photographs,
- synthetic eyewitness accounts,
- invented citations,
- fake documents,
- realistic audio recordings,
- apparently authoritative summaries.
The resulting content may appear more complete and credible than older forms of misinformation. Reviews of generative AI misinformation consistently identify its ability to produce persuasive, polished and context-rich false narratives. [ACM Digital Library]dl.acm.orgCCS ConceptsACM Digital LibraryUnderstanding AI-Generated Misinformation and…by J Zhou · 2023 · Cited by 551 — We discuss implications for practit…
The challenge for note writers is that correcting a polished false story often requires substantially more effort than creating it.
What AI-written note drafts might change
One response to the scale problem is to use AI on the correction side as well.
Several research projects have explored systems that help contributors draft notes, organise evidence and identify claims requiring verification. Experimental work suggests that AI-assisted note-writing tools can improve note quality and help less experienced contributors produce corrections comparable to those written by more experienced fact-checkers. [arXiv]arxiv.orgCANote: Empowering Fact-checking Note Writing Through Scaffolded and Provenance-based Human-AI CollaborationJune 5, 2026…
Potential advantages include:
- Faster drafting of notes.
- Automatic extraction of key claims.
- Assistance locating relevant evidence.
- Improved consistency of note structure.
- Greater coverage of emerging misinformation.
Some platforms have also explored allowing AI systems to propose notes that humans subsequently evaluate. Supporters argue that this could dramatically increase correction capacity and reduce response times. [The Washington Post]washingtonpost.comThe Washington Post X will let AI bots fact-check postsIt isn't as crazy as it sounds.July 3, 2025 — X, the social media platform formerly known as Twitter, is expanding its Community Notes fa…
In theory, this creates a more symmetrical contest: AI generates misinformation, while AI helps generate corrections.
The risk of scaling mistakes
However, AI-generated notes introduce their own dangers.
Language models are capable of producing persuasive but inaccurate explanations. They may misunderstand context, cite unreliable sources or generate plausible-sounding errors. Critics of AI-written Community Notes have warned that fluency can be mistaken for accuracy and that automated systems may appear authoritative even when wrong. [cetas.turing.ac.uk]cetas.turing.ac.ukeverything could go wrong xs new ai written community notesEverything that could go wrong with X's new AI-written…2 Jul 2025 — AI chatbots often struggle with nuance and context but are good at…
The broader problem is familiar from other domains. Researchers continue to document fabricated citations, distorted references and confidently presented inaccuracies in AI-generated content. [TechRadar]techradar.comTech Radar A major KPMG report on AI was found to be chock-full of…AI hallucinations Yesterday — A recent investigation by GPTZero hasThe report contained 45 citations, with only five found to be accurate; the rest were either fabricated, distorted, or misleading. GPTZer…
If correction systems rely too heavily on automation, they risk creating a new bottleneck: humans reviewing large volumes of machine-generated notes whose factual quality is uncertain.
Where human judgment remains necessary
The temptation in an AI-heavy environment is to treat misinformation detection as a purely technical problem. In practice, many of the hardest cases remain fundamentally human.
A note often succeeds not because it identifies a factual error, but because it provides context that readers from different viewpoints recognise as useful. Community Notes was designed around this principle, requiring agreement across contributors with differing rating histories rather than simple majority support. [arXiv]arxiv.orgFrom Birdwatch to Community Notes, from Twitter to X13 Oct 2025 — Community Notes (formerly known as Birdwatch) is the first large-s…
Human judgment remains particularly important in three situations.
Context-dependent claims
Many misleading posts are not completely false.
An image may be genuine but years old. A statistic may be accurate but misleadingly framed. A video clip may omit crucial context. Determining what context readers need often requires interpretation rather than binary classification.
Ambiguous evidence
AI detection tools are imperfect and can degrade as generation methods improve. Recent research on multimodal misinformation found that detection systems face growing difficulty distinguishing synthetic from authentic content over time. [arXiv]arxiv.orgThe Synthetic Media Shift: Tracking the Rise, Virality, and Detectability of AI-Generated Multimodal MisinformationApril 15, 2026…
When automated tools disagree, human reviewers must decide how much confidence to place in the available evidence.
Trust and legitimacy
Public correction systems do more than identify inaccuracies. They also signal why a correction should be trusted.
Research suggests that readers often view community-generated explanatory notes as more trustworthy than simple warning labels. [PMC]pmc.ncbi.nlm.nih.govCommunity notes increase trust in fact-checking on social mediaby CP Drolsbach · 2024 · Cited by 95 — Here, we presented Americans wit… That trust depends on visible reasoning, transparent sources and accountable human participation. Fully automated corrections may struggle to achieve the same legitimacy, especially in politically contested or emotionally charged situations.
The likely future: hybrid correction at AI speed
The most plausible response to AI-generated misinformation is neither purely human review nor fully automated moderation. Instead, emerging evidence points toward hybrid systems in which AI assists with detection, triage, evidence gathering and draft generation, while humans retain responsibility for judgment and final approval. [arXiv]arxiv.orgCANote: Empowering Fact-checking Note Writing Through Scaffolded and Provenance-based Human-AI CollaborationJune 5, 2026…
Such systems may help close part of the speed gap between misinformation production and public correction. Yet the underlying asymmetry remains. Generative AI can create content almost instantly, while trustworthy correction still requires verification, deliberation and evidence.
For critical thinkers, that reality carries an important lesson. A visible note beneath a post can be valuable context, but the absence of a note should never be treated as proof that a claim is true. In an environment where AI can generate misleading content at unprecedented scale, correction systems are increasingly essential—but they are also increasingly likely to be outnumbered.
Amazon book picks
Further Reading
Books and field guides related to Can human notes keep up with AI fakes?. Use these as the next step if you want deeper reading beyond the article.
Calling Bullshit
Explains how misleading information scales in data-rich environments.
The Demon-Haunted World
Provides enduring principles for assessing extraordinary claims.
eBay marketplace picks
Marketplace Samples
Example marketplace items related to this page. Use the search link to explore similar finds on eBay.
Endnotes
-
Source: arxiv.org
Title: arXiv Characterizing AI-Generated Misinformation on Social Media
Link: https://arxiv.org/abs/2505.10266Source snippet
Characterizing AI-Generated Misinformation on Social MediaMay 15, 2025...
Published: May 15, 2025
-
Source: arxiv.org
Link: https://arxiv.org/html/2505.10266v1Source snippet
Characterizing AI-Generated Misinformation on Social Media15 May 2025 — Our work: In this study, we characterize AI-generated misinf...
Published: May 2025
-
Source: link.springer.com
Link: https://link.springer.com/article/10.1007/s00146-025-02620-3Source snippet
Generative AI and misinformation: a scoping review of the role...by S Park · 2025 · Cited by 48 — Analyzing 24 empirical studies...
-
Source: arxiv.org
Link: https://arxiv.org/abs/2510.11423Source snippet
Beyond the Crowd: LLM-Augmented Community Notes for Governing Health MisinformationOctober 13, 2025...
Published: October 13, 2025
-
Source: dl.acm.org
Title: CCS Concepts
Link: https://dl.acm.org/doi/fullHtml/10.1145/3544548.3581318Source snippet
ACM Digital LibraryUnderstanding AI-Generated Misinformation and...by J Zhou · 2023 · Cited by 551 — We discuss implications for practit...
-
Source: arxiv.org
Link: https://arxiv.org/abs/2606.07101Source snippet
CANote: Empowering Fact-checking Note Writing Through Scaffolded and Provenance-based Human-AI CollaborationJune 5, 2026...
Published: June 5, 2026
-
Source: cetas.turing.ac.uk
Title: everything could go wrong xs new ai written community notes
Link: https://cetas.turing.ac.uk/news/everything-could-go-wrong-xs-new-ai-written-community-notesSource snippet
Everything that could go wrong with X's new AI-written...2 Jul 2025 — AI chatbots often struggle with nuance and context but are good at...
-
Source: techradar.com
Link: https://www.techradar.com/pro/a-major-kpmg-report-on-ai-was-found-to-be-chock-full-of-[ai-hallucinationsSource snippet
The report contained 45 citations, with only five found to be accurate; the rest were either fabricated, distorted, or misleading. GPTZer...
-
Source: arxiv.org
Link: https://arxiv.org/html/2510.09585v2Source snippet
From Birdwatch to Community Notes, from Twitter to X13 Oct 2025 — Community Notes (formerly known as Birdwatch) is the first large-s...
-
Source: arxiv.org
Link: https://arxiv.org/abs/2604.15372Source snippet
The Synthetic Media Shift: Tracking the Rise, Virality, and Detectability of AI-Generated Multimodal MisinformationApril 15, 2026...
Published: April 15, 2026
-
Source: pmc.ncbi.nlm.nih.gov
Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC11212665/Source snippet
Community notes increase trust in fact-checking on social mediaby CP Drolsbach · 2024 · Cited by 95 — Here, we presented Americans wit...
-
Source: dl.acm.org
Link: https://dl.acm.org/doi/10.1145/3686967Source snippet
the Roll-Out of Community Notes Reduce Engagement...8 Nov 2024 — In this paper, we perform a large-scale empirical study to analyze whet...
-
Source: misinforeview.hks.harvard.edu
Link: https://misinforeview.hks.harvard.edu/article/people-are-more-susceptible-to-misinformation-with-realistic-ai-synthesized-images-that-provide-strong-evidence-to-headlines/Source snippet
Misinformation ReviewPeople are more susceptible to misinformation with...November 11, 2025 — by S Guo · 2025 · Cited by 2 — In a pre-re...
Published: November 11, 2025
-
Source: livescience.com
Title: Live Science AI is getting better and better at generating faces
Link: https://www.livescience.com/health/psychology/ai-is-getting-better-and-better-at-generating-faces-but-you-can-train-to-spot-the-fakesSource snippet
These faces often appear more realistic than actual human faces, a phenomenon termed "hyperrealism." In a series of experiments, super re...
-
Source: washingtonpost.com
Title: The Washington Post X will let AI bots fact-check posts
Link: https://www.washingtonpost.com/politics/2025/07/03/x-community-notes-ai-[fact-checksSource snippet
It isn't as crazy as it sounds.July 3, 2025 — X, the social media platform formerly known as Twitter, is expanding its Community Notes fa...
Published: July 3, 2025
-
Source: pmc.ncbi.nlm.nih.gov
Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC13057141/Source snippet
by W Glauser · 2026 — Misinformation is increasingly spread with single clicks, bots, and artificial intelligence (AI) deepfakes. AI-g...
-
Source: pmc.ncbi.nlm.nih.gov
Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC12924558/Source snippet
AI and health misinformation - PMC - NIHby HR Saeidnia · 2026 · Cited by 9 — This systematic review synthesizes evidence on how generativ...
-
Source: misinforeview.hks.harvard.edu
Link: https://misinforeview.hks.harvard.edu/article/misinformation-reloaded-fears-about-the-impact-of-generative-ai-on-misinformation-are-overblown/Source snippet
reloaded? Fears about the impact of...by FM Simon · 2023 · Cited by 198 — We argue that current concerns about the effects of generative...
-
Source: tsjournal.org
Link: https://tsjournal.org/index.php/jots/article/view/139/57Source snippet
How Effective is Community Notes in Moderating Content across Twitter? The Future of Crowdsourced Content Moderation.R...
Additional References
-
Source: ntia.gov
Link: https://www.ntia.gov/programs-and-initiatives/artificial-intelligence/open-model-weights-report/risks-benefits-of-dual-use-foundation-models-with-widely-available-model-weights/societal-risks-well-beingSource snippet
Societal Risks and Well-BeingMisinformation reloaded? Fears about the impact of generative AI on misinformation are overblown. Harvard Ke...
-
Source: researchgate.net
Link: https://www.researchgate.net/publication/396457977_From_Birdwatch_to_Community_Notes_from_Twitter_to_X_four_years_of_community-based_content_moderationSource snippet
(PDF) From Birdwatch to Community Notes, from Twitter to X10 Oct 2025 — Community Notes (formerly known as Birdwatch) is the first large...
-
Source: liverpool.ac.uk
Link: https://www.liverpool.ac.uk/research/frontiers/ai-for-life/ai-innovation/safer-world/challenge-of-misinformation/Source snippet
University of LiverpoolGenerative AI and the challenge of misinformation | ResearchStudies have shown that GAI systems can fake academic...
-
Source: youtube.com
Link: https://www.youtube.com/watch?v=rnNcmW0b5U0Source snippet
Addressing misinformation and deepfakes in the mediaThis webinar is being organized as part of the activities of the AI and Multimedia st...
-
Source: orbilu.uni.lu
Link: https://orbilu.uni.lu/bitstream/10993/59462/1/3686967.pdfSource snippet
the Roll-Out of Community Notes Reduce Engagement...by Y CHUAI · 2024 · Cited by 102 — In this paper, we perform a large-scale empirical...
-
Source: semanticscholar.org
Link: https://www.semanticscholar.org/paper/From-Birdwatch-to-Community-Notes%2C-from-Twitter-to-Mohammadi-Chinichian/d9e71e044c4f95c74fc0c35f9e703e8c9beb35d9Source snippet
Notes, and a major curated dataset and accompanying source code to support...
-
Source: instagram.com
Link: https://www.instagram.com/reel/DEpbolcuwZz/Source snippet
ecking re: Meta's move towards community notes...
-
Source: europarl.europa.eu
Title: EPRS BRI(2025)779259 EN
Link: https://www.europarl.europa.eu/RegData/etudes/BRIE/2025/779259/EPRS_BRI%282025%29779259_EN.pdfSource snippet
manipulation in the age of generative artificial...11 Dec 2025 — It will assist deployers using deepfakes or AI-generated content to cle...
-
Source: youtube.com
Link: https://www.youtube.com/watch?v=H_ZO_F4u0fsSource snippet
4 Justin Ling: We Lost the Fight Against Disinformation | Forum TV...
-
Source: firstdraftnews.org
Title: What is Birdwatch?
Link: https://firstdraftnews.org/articles/what-is-birdwatch/Source snippet
First Draft News4 Feb 2021 — Birdwatch is a new program launched by Twitter to combat misinformation on the platform, using volunteers to...
Topic Tree