What an AI Assistant Actually Is (and Isn't)

If you've used ChatGPT, Claude or Gemini, you're in good company. The Thomson Reuters · 2026 AI in Professional Services Report (opens in a new tab) found that 41% of law firms and 47% of corporate legal departments say their legal teams are using GenAI, up from 28% and 23% respectively in 2025. Broader surveys report even higher numbers. The figure depends heavily on how the question is asked: survey design moves the answer from 26% to 92%.

Anyone who has used these tools for real work has likely seen two things. The output is often useful: a workable first draft of a clause, a decent summary of a long agreement. And it is sometimes wrong in ways that are hard to spot, because a wrong answer reads exactly like a right one. A summary can skip the one provision that matters; a case citation can look real and not exist.

Both come from the same place. Once you understand what an AI assistant is doing under the hood, you'll know when to rely on it and when to check its work.

It predicts text; it doesn't look things up

The engine behind every AI assistant is a large language model (LLM). An LLM is trained on an enormous amount of text, including books, websites, legislation, commentary and code. From that training it learns the patterns of language: which words, arguments and structures tend to follow which.

When you ask it a question, it doesn't search a database for the answer. It generates a response one piece at a time, each time producing what is most plausible given everything before it.

That is why it writes so well. It has absorbed the style of millions of well-drafted documents, including many contracts, memos and judgments.

It is also why it can be wrong without knowing it. A plausible sentence and a true sentence are not the same thing. Ask for a case supporting your argument and the model will produce something that looks exactly like a case supporting your argument: a realistic name, a court, a year, a pinpoint citation, like Harlow v. Meridian Holdings, 758 F.3d 412, 419 (2d Cir. 2014)Invented. Sometimes the case is real. Sometimes it has been assembled from patterns and never existed. This is called a hallucination, and the model's tone gives you no clue which one you're looking at.