Official launch partners
Chapter 2 · Can I trust it?
00 Introduction 3 lessons
  1. 0.1 Who this guide is for and how to read it
  2. 0.2 The eight questions every lawyer asks
  3. 0.3 Survival glossary
01 How can AI help my practice and day-to-day work? 5 lessons
  1. 1.1 What is AI really?
  2. 1.2 Chat vs. agent
  3. 1.3 Where skills, MCP and plugins fit in
  4. 1.4 What lawyers are actually using AI for right now
  5. 1.5 Does it really save time?
02 Can I trust it? 5 lessons
  1. 2.1 Hallucinations
  2. 2.2 What is the answer based on?
  3. 2.3 When the model tells you what you want to hear
  4. 2.4 What a benchmark score really tells you
  5. 2.5 Test it on your own matters
03 Am I allowed to use it? 7 lessons
  1. 3.1 The three questions behind the question
  2. 3.2 What happens to the documents I upload?
  3. 3.3 Who can access, store or reuse my data?
  4. 3.4 Anonymisation and pseudonymisation
  5. 3.5 Professional rules by jurisdiction
  6. 3.6 Sovereignty and compliance
  7. 3.7 Other risks
04 How do I choose the best tools? 6 lessons
  1. 4.1 Same brain, different bodies
  2. 4.2 Subscription vs. API access
  3. 4.3 Legal AI tool or general-purpose assistant?
  4. 4.4 Open-source vs. closed, from the buyer's seat
  5. 4.5 Panorama of tools
  6. 4.6 Questions to ask before choosing a legal AI tool
05 How do we make it work across the firm or legal team? 7 lessons
  1. 5.1 Why one enthusiast is not an adoption strategy
  2. 5.2 Choosing a first pilot
  3. 5.3 Training the team
  4. 5.4 Measuring time saved and quality
  5. 5.5 Who maintains the tools and the shared know-how?
  6. 5.6 Change management
  7. 5.7 Working with IT, security and procurement
06 How do I get better results? 3 lessons
  1. 6.1 Give it better context
  2. 6.2 Turning your methods into reusable instructions
  3. 6.3 Watch out for AI slop
07 What does this mean for my career and my firm? 4 lessons
  1. 7.1 Which skills should lawyers develop?
  2. 7.2 What does a legal engineer do?
  3. 7.3 How AI may change fees, staffing and client expectations
  4. 7.4 What an AI-native firm might look like
08 Where and how do I start now? 4 lessons
  1. 8.1 Starter kits by profile
  2. 8.2 Classic mistakes to avoid
  3. 8.3 Section 1 recap
  4. 8.4 Staying current without drowning

Lesson 2.1 Can I trust it? 7 min read

Hallucinations (Rarer, but Subtler)

AC 1951QB 2006WLR 1980All ER 1935Ch 1941KB 1956Lloyd's 2019Cr App R 1970AC 1932QB 1976WLR 1983All ER 1998Ch 1988KB 1933Lloyd's 1969EWCA 812AC 1953QB 1973WLR 1961All ER 2012Ch 2005KB 1970Lloyd's 2017Cr App R 1970AC 2003

In 2023, a New York lawyer filed a brief citing six cases that did not exist. ChatGPT had written them. When he asked it whether they were real, it said yes. The judge in Mata v. Avianca fined him, a colleague and their firm $5,000.

Since then, models have made far fewer of these errors. A 2026 study ran eight generations of ChatGPT on the same legal drafting tasks. The 2023 model invented about one case citation in four; the late-2025 model, arXiv Who Checks the Citations?, Jun 2026 (opens in a new tab).

Yet more of these errors reach the courts every quarter. Far more people now use AI to write legal documents, including many people with no lawyer, and not all of them check the result.

Court decisions dealing with AI-invented content, per quarter

Involving lawyers Other parties, mostly self-represented litigants
0
150
300
450

Share of invented case citations in ChatGPT's default model, at release

0%
15%
30%

Courts see more AI-invented cases, even as models invent fewer.

Court decisions: Damien Charlotin, AI Hallucination Cases database (CC BY 4.0), October 2026, courts worldwide. Citation rates: Liu, Stammbach and Henderson, Who Checks the Citations? (2026), 92 legal drafting prompts, no web search. Lawve analysis.

Each of these decisions involves a : a false statement written with the same confidence as a true one. The errors are rarer. They are also harder to see.

What remains is subtler

An invented case is the error everyone has heard of. It is not the only one. Before reading on, check the citations in this extract from a brief.

Click a citation you would not rely on.

A manufacturer owes a duty of care to the person who ends up using its product.1 As Lord Atkin put it, a manufacturer “is answerable for every injury its products cause”.2 A duty of care arises whenever harm to the claimant was foreseeable.3 The same principle applies to food packaging.4

  1. 1 Donoghue v Stevenson [1932] AC 562
  2. 2 Donoghue v Stevenson [1932] AC 562, 599
  3. 3 Caparo Industries plc v Dickman [1990] 2 AC 605
  4. 4 Hartley v Brennan Foods Ltd [1998] 1 WLR 1123

Courts see the same three kinds of error. Most decisions still involve an invented case. Many also involve a real case cited for something it doesn't say, or a quotation the court never wrote.

  • Invented authority

    A case, statute or article that doesn't exist.

    89%

  • Real authority, wrong content

    The source exists but doesn't say what it is cited for.

    51%

  • Invented quotation

    Words in quotation marks that the source never used.

    38%

Fewer decisions involve an invented case.

Damien Charlotin, AI Hallucination Cases database, October 2026, decisions involving lawyers. A decision can include several types of error, so the shares add up to more than 100%. Lawve analysis.

The last two are harder to catch. The case exists and the citation checks out. The mistake is in what the brief says the case holds, or in the words it puts in the judge's mouth. Some briefs now contain only these errors, and no invented case at all.

One likely reason is that more AI tools now search real case law before answering: they invent fewer cases, but they can still misread the ones they find. Checking that a source exists is not the same as checking that it says what the answer claims.

Why errors remain

Three things explain most of what is left.

It writes what sounds right. As lesson 1.1 explained, a model writes the most plausible next words. It knows exactly what a citation looks like, so it can always write one.

It answers when a name rings a bell. Researchers at Anthropic watched what happens inside Claude when it is asked about a person Anthropic Tracing the thoughts of a language model, Mar 2025 (opens in a new tab). Asked about Michael Jordan, it recognises the name and answers. Asked about a name it has never seen, it says it doesn't know. The trouble is a name in between: one that sounds familiar, but about which the model knows nothing. The recognition kicks in, the "I don't know" does not, and the model makes up an answer. A rarely cited case can be exactly that kind of name.

It was taught to guess. For years, models were not trained to say "I don't know". They were trained and ranked on tests that gave a point for a right answer and nothing for admitting doubt, so guessing always paid arXiv Why Language Models Hallucinate, Sep 2025 (opens in a new tab). Labs now train some models to decline when they are unsure. Others still push for more right answers and accept more wrong ones. A model that knows more is not always a model that invents less.

0%
20%
40%
60%
80%
↑ Wrong guesses when it doesn't know (lower is better) → Questions answered correctly (higher is better)

Each dot is a model. The best place is bottom right: many right answers, few wrong guesses.

Artificial Analysis, AA-Omniscience, October 2026: 6,000 deliberately hard factual questions, including law, answered without web search.

What this means for your work

Hallucinations are much rarer than in 2023. But newer models cite more cases per answer, and less famous ones arXiv Who Checks the Citations?, Jun 2026 (opens in a new tab), so even a small error rate leaves something to find. The risk sits in the precise details.

Lower riskCheckCheck first

Manufacturers owe a duty of care to the people who end up using their products, even without a contract. The leading case is Donoghue v Stevenson, decided by the House of Lords in 1932. Mrs Donoghue claimed £500 after finding the remains of a snail in her ginger beer. Lord Atkin said: “You must take reasonable care to avoid acts or omissions which you can reasonably foresee would be likely to injure your neighbour”.

  • General explanation

    Models are at their best here. Read it as you would a textbook: critically, but it is rarely invented.

  • Case name

    Famous cases are well known to models; obscure ones are where names get invented. Look it up in a database.

  • Numbers

    A year, a court or an amount can drift without anyone noticing. Find each one in the source.

  • Quotation

    A quotation can be altered or invented even when the case is real. Compare it word for word with the judgment.

The general explanation is the safest part. Precise details are where errors hide.

Illustrative answer about a real case. Every detail in it is correct; the colours show how much each kind of detail needs checking, not whether this one is wrong.

  • Check the precise details, and read the passage behind each citation: a real source can still be misread.
  • Prefer answers built on your own documents, where you can trace each statement back to a passage.
  • The responsibility stays with you. In High Court Ayinde v Haringey, Jun 2025 (opens in a new tab), the court held that lawyers who use AI must check its output against authoritative sources, and referred the lawyers involved to their regulators.

Next, lesson 2.2 looks at what an answer is based on: the model's memory, your documents or a live search.