What the Answer Is Based On (Memory, Documents & Search)

A client wants to end a supplier contract early. You ask an AI assistant whether the client can. With nothing else to go on, it answers from the general law: Yes, on reasonable notice. That is a fair rule for a contract that says nothing on the point. This one doesn't.

You attach supply-agreement.pdf and ask again. The answer flips: No, not before March 2027. Clause 12.1 allows thirty days' notice, but Schedule 2 bars it in the first year, and leaving early after that costs three months' fees.

Same model, same question. The only difference is what it had in front of it. What the model reads matters more than what it remembers, and you decide what it reads.

Three sources, one answer

An assistant can draw on three sources. The first is its 1memory: what the model learned in training. Ask it something with nothing attached, and that is all it has. It is enough to explain a rule, structure an argument or draft a clause. But it does not look anything up, and what it knows stops at a date (lesson 1.1).

The second is the 2documents you give it. Attach the client's contract or paste the other side's letter, and it answers from facts no model has ever seen. It can also point to the passage it relied on.

The third is a 3search. Turn it on, and it looks for web pages or judgments, reads some of them and answers from what it found. Legal research tools do the same in their own database of case law and legislation.

The three add up. When the model reads your contract, it uses what it learned to understand what a termination clause does. Your documents bring the facts of the matter. A search finds the law and the authorities to cite.

Adding sources makes a large difference. On legal research questions written by lawyers, the best model got 8.7% of answers fully right from memory alone, and 42.9% once it could search case law and the web arXiv · Legal Research Bench, Sep 2026 (opens in a new tab). Every model did better with tools than any model without them. For anything you will rely on, don't leave the model with only its memory.

From invention to omission

A source does not make the model error-free. It changes the error. Without one, the model may invent (lesson 2.1). With a source, it may leave out what matters.

In a 2026 study, 12 models summarised the full files of 100 US civil rights cases. Compared with the summaries lawyers had written, their main failure was leaving points out, not inventing them: even the best model covered fewer than 6 in 10 of the points the lawyers included arXiv · Gavel, EMNLP 2026 (opens in a new tab).

Omission is harder to catch than invention. You catch an invented case by looking it up. A missing condition leaves nothing to look up: an answer built on clause 12.1 gives no hint that Schedule 2 exists. You only find it by reading the source. When the answer has a source, check what is missing, not only what is there.

Attaching is not reading

Attaching a file feels like handing it to a colleague who will read it. With a short file, that is roughly what happens. With a long one, or a whole bundle, it rarely is. Most assistants now work as agents: they have search tools, and they use them on your documents as they would on the web. They search the passages that seem relevant, read those, and answer from them.

There are good reasons for this. A model only works from what fits in its context window, and the largest hold about 1,500 pages (lesson 1.1). A data room of a few hundred contracts holds far more.

Even when everything fits, reading it all has a price. The model processes every word you send on powerful computers that someone pays to run, so more text means a longer wait and a higher bill.

Results also get worse as the input grows. In a 2026 test, recent frontier models missed a single risky step 2 to 30 times more often when it was buried after about 1,200 pages of routine log than when it stood on its own arXiv · Classifier Context Rot, May 2026 (opens in a new tab). So agents are built and instructed to search first.

When the search misses, the passage never comes back, and the model answers from an incomplete file without knowing it. Whether the right passage comes back depends on the search tool, and the one most likely to stay behind is one that matters but looks unlike your question. Here is what that does to the client's contract.

Reading everything is not a simple fix either. Researchers who tried both on the case files found a trade-off: searching was cheaper but found less, while reading every file in full found more but cost more and added errors of its own arXiv · Gavel, EMNLP 2026 (opens in a new tab). The answer is only as complete as what the search brought back.

What this means for your work

You cannot see inside the search, but you can steer it. Start by choosing what the model reads: the whole contract for a question about the contract, and a search in a legal database for the law, preferring the text of the law or the judgment to a commentary. For a large bundle, give it the documents that matter, not everything you have (lesson 1.1).

Then make your question do the work. Name the parts that matter, such as schedules, definitions and amendments, so the search brings them back even if they look nothing like your question. Ask for the passages it relied on, so you can see what came back. And before relying on the answer, ask what else bears on the question. Name what matters, and ask to see what came back. Try each habit on the client's question.

Next, lesson 2.3, "When the model tells you what you want to hear", looks at why a model can sound sure of an answer, then drop it as soon as you push back.