Within Source Tracing
Why fake AI citations reached the courtroom
Court filings show why fake AI citations are not harmless formatting errors when professional duties depend on verification.
On this page
- What made legal hallucinations easy to test
- The professional duty to verify AI research
- Lessons for non lawyers tracing sources
Page outline Jump by section
Introduction
Legal AI hallucinations became a public warning about citation risk because courts provided something rare: a setting where citations could be checked immediately against authoritative records. When lawyers submitted briefs containing cases, quotations and rulings that did not exist, the problem was not an abstract concern about AI accuracy. It became a visible failure of evidence verification. Court filings transformed AI hallucinations from a technical curiosity into a professional and ethical issue because legal arguments depend on traceable sources, and judges, opposing counsel and court staff routinely verify them. [UC Berkeley Law]law.berkeley.eduUC Berkeley Law Mata vAvianca, Inc., 678 F.Supp.3d 443 (2023)judicial opinions with fake quotes and citations created by the artificial intelligence tool ChatG…
Within the broader challenge of critical thinking in the age of social media and AI, these incidents demonstrated a simple lesson: a citation is only useful if it leads to a real source that supports the claim being made. The courtroom exposed what happens when that chain of verification breaks.
What made legal hallucinations easy to test
Unlike many online claims, legal citations point to specific and searchable materials. A court opinion either exists or it does not. A quotation either appears in the cited case or it does not. This made legal hallucinations unusually easy to detect.
The best-known example was the 2023 case Mata v. Avianca. Lawyers representing a plaintiff submitted a filing containing multiple cases generated by ChatGPT. Several cited decisions were entirely fictitious. When opposing counsel and the court attempted to locate the authorities, they could not be found. Judge P. Kevin Castel later sanctioned the lawyers, noting that the filing relied on non-existent judicial opinions and fabricated legal analysis. [UC Berkeley Law+2Justia]law.berkeley.eduUC Berkeley Law Mata vAvianca, Inc., 678 F.Supp.3d 443 (2023)judicial opinions with fake quotes and citations created by the artificial intelligence tool ChatG…
What made the incident memorable was not merely that the AI was wrong. Humans had opportunities to discover the error. The lawyers asked the system whether the cases were real and accepted its reassurance rather than checking the cited authorities through standard legal research tools. The failure therefore revealed a broader risk: AI can generate sources that look authentic enough to discourage further scrutiny. [UC Berkeley Law]law.berkeley.eduUC Berkeley Law Mata vAvianca, Inc., 678 F.Supp.3d 443 (2023)judicial opinions with fake quotes and citations created by the artificial intelligence tool ChatG…
Subsequent incidents reinforced the pattern. Michael Cohen’s legal team submitted filings containing fictitious cases reportedly generated through an AI system, creating procedural problems even though the judge ultimately declined to impose sanctions for bad faith. The issue again centred on verification rather than technology alone. [OECD.AI+2The Guardian]oecd.aining fake legal case citations generated by Google Bard, an AI chatbot…
Researchers studying legal AI have found that hallucinations are particularly concerning because legal questions often involve highly specific, verifiable authorities. Studies of large language models answering legal questions have documented substantial rates of fabricated or inaccurate legal references, even when responses appear confident and professionally written. [arXiv]arxiv.orgLarge Legal Fictions: Profiling Legal Hallucinations in Large Language ModelsJanuary 2, 2024…
The professional duty to verify AI research
The legal profession reacted strongly because lawyers already have a duty to ensure that factual statements and cited authorities are accurate. AI did not create this obligation; it merely created a new way to fail it.
In the Mata sanctions order, the court emphasised that using AI is not inherently improper. The problem arose because attorneys failed to perform their traditional gatekeeping role and verify the accuracy of material submitted to the court. [Relativity]relativity.comai case law update the lamborghini doctrine of hallucinationsAI Case Law Update: The Lamborghini Doctrine of…7 Aug 2025 — Since Judge Castel's May 2023 order to show cause in Mata, the…
This distinction is important. Courts have generally not treated AI use itself as misconduct. Instead, judges have focused on whether lawyers exercised reasonable professional judgement before filing documents. The legal system expects practitioners to know the sources they rely upon, regardless of whether those sources were found through a database search, a junior colleague, or an AI assistant. [Esquire Deposition Solutions]esquiresolutions.comfederal court turns up the heat on attorneys using chatgpt for researchAvianca court imposed monetary fines and other remedial measures against the offending attorneys, citing Rule 11 of the Federal…Read more…
As more incidents accumulated, judicial patience appeared to decrease. Courts in the United States and elsewhere have imposed fines, sanctions and disciplinary measures where lawyers submitted fabricated citations linked to AI-generated research. In 2026, a federal judge in Mississippi sanctioned and disqualified attorneys from both sides of a dispute after unverified AI-generated authorities appeared in filings. The judge stressed that responsibility remained with the lawyers who signed the documents. [Reuters+2Business Insider]reuters.comJudge rules both sides in lawsuit misused AI, disqualifies lawyersDistrict Judge in Mississippi, Sharion Aycock, has disqualified all attorneys involved in a contract dispute case after discovering both…
The problem has also spread beyond a single jurisdiction. UK judges have warned legal professionals about AI-generated false authorities after multiple cases involving fictitious citations. Senior judges described the issue as a threat to the proper administration of justice because courts depend on accurate representations of the law. [The Guardian]theguardian.comIn one prominent case against the Qatar National Bank, 18 out of 45 cited cases were entirely fictitious, generated via AI tools used by…
Why the courtroom became a warning for everyone else
Legal cases attracted attention because they created unusually clear demonstrations of source failure.
Many online misinformation disputes involve interpretation. People can disagree about context, meaning or significance. By contrast, a non-existent court decision is easy to evaluate. Either the case exists in legal records or it does not. This binary quality made legal hallucinations highly visible examples of a broader problem affecting AI-assisted research.
The incidents also exposed a common misunderstanding about citations. Many users assume that a reference automatically increases credibility. The courtroom showed that citations themselves can be fabricated. A document filled with case names, docket numbers and legal language may still rest on invented authorities if nobody checks the underlying sources. [UC Berkeley Law]law.berkeley.eduUC Berkeley Law Mata vAvianca, Inc., 678 F.Supp.3d 443 (2023)judicial opinions with fake quotes and citations created by the artificial intelligence tool ChatG…
Legal hallucinations therefore became a practical demonstration of source tracing. They showed that verification cannot stop at the appearance of evidence. The user must be able to follow the citation to a real document and confirm that it supports the claim.
Lessons for non-lawyers tracing sources
Most people will never file a court brief, but the lessons from these cases apply widely to AI-assisted work.
Treat citations as claims, not proof. A reference should trigger verification rather than trust. If a source cannot be located, confidence in the associated claim should drop sharply.
Check the original material whenever stakes are high. Legal professionals learned that relying on AI summaries alone is risky. The same principle applies to health information, financial decisions, academic work and public policy debates.
Look for independently verifiable sources. Courts exposed hallucinations because legal authorities are archived and searchable. Readers should prefer sources that can be inspected directly rather than claims that depend entirely on an AI system’s description.
Be cautious when the AI sounds unusually certain. Research on legal hallucinations suggests that large language models often produce confident answers even when they are incorrect and may not reliably recognise their own errors. [arXiv]arxiv.orgLarge Legal Fictions: Profiling Legal Hallucinations in Large Language ModelsJanuary 2, 2024…
Remember that human responsibility remains. The courtroom cases did not end with judges blaming software. Sanctions fell on the people who submitted the documents. That principle extends beyond law: AI may generate information, but users remain responsible for deciding whether the information deserves trust. [Relativity+2Reuters]relativity.comai case law update the lamborghini doctrine of hallucinationsAI Case Law Update: The Lamborghini Doctrine of…7 Aug 2025 — Since Judge Castel's May 2023 order to show cause in Mata, the…
The significance of the legal cases was therefore not simply that AI invented citations. It was that a profession built around evidence verification publicly demonstrated how easily authoritative-looking references can bypass human scepticism. In an era when AI systems increasingly mediate access to information, those courtroom failures became a vivid reminder that every citation must lead back to a real, checkable source.
Amazon book picks
Further Reading
Books and field guides related to Why fake AI citations reached the courtroom. Use these as the next step if you want deeper reading beyond the article.
Calling Bullshit
Provides practical frameworks for checking evidence and challenging unsupported claims.
Artificial Intelligence and Legal Analytics
First published 2017. Subjects: Artificial intelligence, Computers, law and legislation, Law, Methodology, Technological innovations.
Thinking, Fast and Slow
Explains judgment errors that contribute to overreliance on AI-generated outputs.
Bad Science
Illustrates the consequences of trusting unsupported claims without verification.
eBay marketplace picks
Marketplace Samples
Example marketplace items related to this page. Use the search link to explore similar finds on eBay.
Endnotes
-
Source: law.berkeley.edu
Title: UC Berkeley Law Mata v
Link: https://www.law.berkeley.edu/wp-content/uploads/archive/2025/12/Mata-v-Avianca-Inc.pdfSource snippet
Avianca, Inc., 678 F.Supp.3d 443 (2023)judicial opinions with fake quotes and citations created by the artificial intelligence tool ChatG...
-
Source: law.justia.com
Link: https://law.justia.com/cases/federal/district-courts/new-york/nysdce/1%3A2022cv01461/575368/54/Source snippet
Mata v. Avianca, Inc., No. 1:2022cv01461 - Document 54...22 Jun 2023 — The narrative leading to sanctions against Respondents incl...
-
Source: oecd.ai
Link: https://oecd.ai/en/incidents/2024-03-20-d459Source snippet
ning fake legal case citations generated by Google Bard, an AI chatbot...
-
Source: arxiv.org
Link: https://arxiv.org/abs/2401.01301Source snippet
Large Legal Fictions: Profiling Legal Hallucinations in Large Language ModelsJanuary 2, 2024...
Published: January 2, 2024
-
Source: arxiv.org
Link: https://arxiv.org/abs/2405.20362 -
Source: relativity.com
Title: ai case law update the lamborghini doctrine of hallucinations
Link: https://www.relativity.com/blog/ai-case-law-update-the-lamborghini-doctrine-of-hallucinations/Source snippet
AI Case Law Update: The Lamborghini Doctrine of...7 Aug 2025 — Since Judge Castel's May 2023 order to show cause in Mata, the...
Published: May 2023
-
Source: reuters.com
Title: Judge rules both sides in lawsuit misused AI, disqualifies lawyers
Link: https://www.reuters.com/legal/litigation/judge-rules-both-sides-lawsuit-misused-ai-disqualifies-lawyers-2026-06-09/Source snippet
District Judge in Mississippi, Sharion Aycock, has disqualified all attorneys involved in a contract dispute case after discovering both...
-
Source: arxiv.org
Link: https://arxiv.org/abs/2603.15937 -
Source: reuters.com
Title: US appeals court fines lawyers $30,000 in latest AI-related sanction A U.S
Link: https://www.reuters.com/legal/litigation/us-appeals-court-fines-lawyers-30000-latest-ai-related-sanction-2026-03-16/Source snippet
federal appeals court has fined attorneys Van Irion and Russ Egli $30,000 for submitting legal filings containing over two dozen fake cas...
-
Source: united.com
Link: https://www.united.com/Source snippet
United AirlinesFind the latest travel deals on flights, hotels and rental cars. Book airline tickets and MileagePlus award tickets to wor...
-
Source: reuters.com
Title: ai hallucinations court papers spell trouble lawyers 2025 02 18
Link: https://www.reuters.com/technology/artificial-intelligence/ai-hallucinations-court-papers-spell-trouble-lawyers-2025-02-18/Source snippet
AI 'hallucinations' in court papers spell trouble for lawyersFeb 18, 2025 — One of the lawyers admitted in court filings last week that h...
-
Source: reuters.com
Title: michael cohen wont face sanctions after generating fake cases with ai 2024 03 20
Link: https://www.reuters.com/legal/michael-cohen-wont-face-sanctions-after-generating-fake-cases-with-ai-2024-03-20/Source snippet
Michael Cohen will not face sanctions after generating fake...Mar 20, 2024 — The citations to three nonexistent cases that Cohen, a form...
-
Source: theguardian.com
Title: michael cohen trump lawyer fake citations
Link: https://www.theguardian.com/us-news/2023/dec/29/michael-cohen-trump-lawyer-fake-citationsSource snippet
Michael Cohen accidentally gave his lawyer fake citations...Dec 29, 2023 — Former Trump lawyer says he used AI program Google Bard and b...
-
Source: esquiresolutions.com
Title: federal court turns up the heat on attorneys using chatgpt for research
Link: https://www.esquiresolutions.com/federal-court-turns-up-the-heat-on-attorneys-using-chatgpt-for-research/Source snippet
Avianca court imposed monetary fines and other remedial measures against the offending attorneys, citing Rule 11 of the Federal...Read more...
-
Source: businessinsider.com
Link: https://www.businessinsider.com/mississippi-judge-removes-lawyers-lawsuit-ai-hallucinations-court-filings-2026-6Source snippet
U.S. District Judge Sharion Aycock sanctioned four attorneys involved in a contractual dispute for submitting briefs containing bogus cit...
-
Source: theguardian.com
Link: https://www.theguardian.com/technology/2025/jun/06/high-court-tells-uk-lawyers-to-urgently-stop-misuse-of-ai-in-legal-workSource snippet
In one prominent case against the Qatar National Bank, 18 out of 45 cited cases were entirely fictitious, generated via AI tools used by...
-
Source: damiencharlotin.com
Link: https://www.damiencharlotin.com/hallucinations/?page=18&parties=Pro+Se+Litigant&period_idx=0&q=&sort_by=-dateSource snippet
in cases where generative AI produced hallucinated content – typically fake citations, but also other types...Read more...
-
Source: theguardian.com
Link: https://www.theguardian.com/technology/2023/jun/23/two-us-lawyers-fined-submitting-fake-court-citations-chatgptSource snippet
Two US lawyers fined for submitting fake court citations...23 Jun 2023 — A US judge has fined two lawyers and a law firm $5,000 (£3,935)...
Additional References
-
Source: pcgamer.com
Link: https://www.pcgamer.com/software/ai/both-lawyers-in-case-use-hallucinating-ai-causing-judge-to-throw-up-hands-bar-them-for-2-years-fine-everybody-and-call-the-whole-thing-off-for-60-days/Source snippet
Attorneys Kathleen M. Wilson and Kathryn Y. Williams used generative AI and did not verify the fictitious legal references it produced. T...
-
Source: foley.com
Link: https://www.foley.com/de/insights/publications/2024/02/uspto-warns-against-blind-reliance-artificial-intelligence/Source snippet
USPTO Warns Against Blind Reliance on Artificial IntelligenceOf particular concern should be citations to legal cases, which should be th...
-
Source: abajournal.com
Link: https://www.abajournal.com/news/article/fake-citations-in-legal-brief-were-generated-by-google-bard-ai-program-says-ex-trump-lawyer-michael-cohenSource snippet
Fake citations in legal brief were generated by Google...Jan 2, 2024 — Former lawyer Michael Cohen has informed a federal judge that the...
-
Source: medium.com
Link: https://medium.com/%40kyle-t-jones/when-ai-goes-wrong-in-court-what-every-lawyer-needs-to-know-about-the-mata-v-avianca-case-ba3575fe328eSource snippet
When AI Goes Wrong in Court: What Every Lawyer Needs...The Mata v. Avianca case changed legal practice. Before June 2023, using AI for l...
Published: June 2023
-
Source: reason.com
Link: https://reason.com/volokh/2024/03/20/no-sanctions-in-michael-cohen-hallucinated-citations-matter/ -
Source: getvoibe.com
Link: https://www.getvoibe.com/resources/ai-hallucinations-law-firms/Source snippet
Avianca first sanctioned ChatGPT-fabricated citations in June 2023, at least eight appellate and trial rulings have imposed fines...Rea...
Published: June 2023
-
Source: goldbergsegalla.com
Link: https://www.goldbergsegalla.com/app/uploads/2023/10/Fake-Cases-Real-Consequences-Misuse-of-ChatGPT-Christoper-F.-Lyon-NY-Litigator.pdfSource snippet
Avianca, Inc., 2023 WL 4138427 (S.D.N.Y. June 22. 2023). The district court finally issued its ruling on sanctions...Read more...
-
Source: forbes.com
Link: https://www.forbes.com/sites/mollybohannon/2023/06/08/lawyer-used-chatgpt-in-court-and-cited-fake-cases-a-judge-is-considering-sanctions/Source snippet
Lawyer Used ChatGPT In Court—And Cited Fake Cases. A...8 Jun 2023 — The attorney said in a filing that he didn't understand ChatGPT “was...
-
Source: incidentdatabase.ai
Link: https://incidentdatabase.ai/cite/541/Source snippet
Avianca, Inc. used ChatGPT for research. ChatGPT hallucinated court cases, which the lawyer then presented in court.Read more...
-
Source: seyfarth.com
Title: update on the chatgpt case counsel who submitted fake cases are sanctioned
Link: https://www.seyfarth.com/news-insights/update-on-the-chatgpt-case-counsel-who-submitted-fake-cases-are-sanctioned.htmlSource snippet
Update on the ChatGPT Case: Counsel Who Submitted...26 Jun 2023 — On June 8, 2023, the judge in the Mata case held a hearing on the issu...
Published: June 8, 2023
Topic Tree