Legal skills in the genAI age, Part 3: Can you (still) smell when AI is wrong?

Why sensing AI's failures, and understanding them, now matters more than skill at using it.

TL;DR: Blindsided by genAI, the traditionally tech-averse legal profession is rushing to put it to work before figuring out whether lawyers can still smell trouble. Yet the productivity bet on AI depends on users knowing what to delegate, trust, and verify. This is a gamble. Lawyers' instincts were formed on human work, while AI produces polished, confident, sycophantic work that can fool even seasoned counsel. Lawyers must retrain those instincts for machine work and develop enough technical and systems understanding to catch what instinct misses. AI literacy is about putting both to work in sound judgments about AI. Not every lawyer will get there, and none can get there alone. Firms must therefore find and cultivate those who can, give them a place to learn and even build, and maintain the checks and records no lawyer alone can keep. How to teach that literacy, and to whom, is the billion-dollar open question.

In Parts 1 and 2, we watched genAI devour legal work from the bottom up and asked how our anxious junior could still form judgment when AI improves her output faster than she grows. Part 2's answer was to make her apprenticeship more productively uncomfortable, handing the machine the busywork and keeping the friction that forms a lawyer.

Now imagine she is a little further along in her pursuit of AI-assisted career excellence. She has grown in judgment (perhaps by following some of my insightful advice). No longer dazzled by the machine, she has learned to tell it "no" with style. She has reached enlightenment in operational AI competency.

Then, after a few model updates, something unsettling happens. She starts running out of objections. Nothing in the machine's work smells off anymore. Perhaps the work really is that good, or perhaps her instincts, formed on human work where sloppy prose often reflected sloppy thinking, are no longer reliable guides. She cannot tell which, and that spells trouble, because the bots only boost productivity if she does not check everything. She must therefore rely on instinct to choose what to verify. But instincts poorly tuned to machine work risk pushing her toward either second-guessing everything or rubber-stamping it all.

This is where AI literacy comes in. Operational competency, the prompting, briefing and steering we explored in Part 2, is about working with the machine. AI literacy is about knowing when and how far to rely on it, which rests on two things. The first is the instinct every good lawyer develops for a matter, recalibrated for machine work. It tells them what to do next, whether the work is good enough, and whether something feels off. The second is enough technical and systems understanding to trace failures and anticipate what instinct cannot catch. That understanding calls for systems thinking more than engineering, which is fortunate for our junior, since she went to law school to become a lawyer, not a legal engineer. The instinct is hers to retrain and the systems thinking hers to acquire, but the surrounding infrastructure is her firm's to supply.

How to develop that literacy, whom to develop it in, and what firms should build around it remain open questions. I doubt anyone has a concrete answer, not because I am the smartest human brooding over this problem, but because few are economically incentivised to solve it. Labs sell models, vendors applications, consultants transformation, law schools degrees, firms hours, and commentators takes. Few sell the slow work of forming a lawyer's judgment. So we will have to work it out ourselves, while the map is still being drawn.

Consider Part 3 my reconnaissance of the terrain. It shows why operational AI competency and instincts built on pre-AI legal experience fall short, what lawyers must learn and unlearn, and what firms must build and institutionalise.

I. When competency falls short and instinct misfires

Our junior stares at the screen. One bot recommends breaching the contract. The other, performance. She gave both the same contract, facts, law, and prompt: "Act as senior counsel with 600 years of practice experience."

She wanted a second opinion before advising the client. Now she has two polished, confident answers pointing in opposite directions, with no obvious reason to prefer either. Her operational skills have given her plausible approaches, but her instincts offer little help deciding which, if either, to trust.

Most lawyers do not run two bots. They skim the output of one and bet on the Fates and their instinct to flag trouble. So, had our junior consulted only the first bot, what would have made her doubt it, if at all?

When competency falls short

Not her prompting skills. Those competency skills matter, but they dominate AI training for lawyers, perhaps because they are easy to teach, certify, and sell. They are also the skills the machines are increasingly learning to perform without us.

The bigger trouble is that knowing how to get an answer does not mean knowing why the machine gave that answer. Our junior may prompt and direct the bots proficiently without appreciating how much the advice depends on her choice of model and seemingly innocuous instructions.

John Rizner and Matthew Krzus gave five frontier models a similar commercial breach of contract hypothetical. Their estimates of the price at which breach pays clustered near the economically rational breach threshold of $2,001, yet their recommendations diverged (Rizner, 2025). Under the baseline prompt, GPT-5 recommended breach every time, while Claude Opus 4.1 did so in only 3.9 per cent of responses. Their bot buddies fell in between (Rizner & Krzus, 2025).

Figure 1. Same math, different advice. All five models put the price that makes breach worthwhile at about $2,001, yet how often they advised breach ranged from 4 to 100 per cent. Breach rates are without a lawyer persona or damages clause. Source: R

This disagreement does not establish which recommendation, if either, is wrong. Our junior can check the arithmetic and challenge the stated reasoning, but the answer alone cannot reveal how the model's tendencies shaped its advice. Worse, her prompting may shift those tendencies in ways she does not anticipate. When the researchers told the models to act as the client's lawyer, Claude Opus 4.1 became 24.4 percentage points more likely to recommend breach, while Gemini 2.5 Pro became 21.0 points less likely. Specifying liquidated damages also shifted recommendations, without producing any alignment.

Figure 2. Same instruction, opposite effects. Change in how often each model advised breach, in percentage points, when told to act as the client's attorney or when the liquidated damages were spelled out. Blue means more breach advice and red less.

Our junior's "600 years of practice experience" prompt may look silly, but by assigning the bot such a persona she hopes to summon an experienced lawyer's wisdom from the depths of the model's weights and harness. It might even be the persona prompting she has learned at her firm's AI bootcamp. Yet such instructions can change the substantive advice, in different directions depending on the model, for reasons she cannot discern. She is doing what she was taught without necessarily understanding what it does.

This is the uncomfortable limit of competency. It cannot tell her which model to trust, and if one model favours breach and another performance, model choice itself becomes part of legal judgment, as Rizner observes. Follow that thought to its rather awkward conclusion, and one of the most consequential judgments on our junior's file may already have been made by whoever signed the firm's AI vendor contract.

When instinct misfires

"Okay, so what? Lawyers disagree and make mistakes all the time. When colleagues do, we examine their reasoning and check the sources ourselves. Why is this an AI literacy problem rather than ordinary legal judgment applied to a fancy new tool?"

Because ordinary legal judgment does not examine everything. It is about knowing when to examine an argument, what to suspect, and how much scrutiny is enough. Seniors do not scrutinise every line of a junior's memo equally. They skim for red flags, such as a suspicious citation, conclusions outrunning their reasoning, or clumsy prose, then investigate. AI productivity depends on making that same skim-and-trust bet with machine work.

But those instincts were formed on humans. We judge colleagues by their competence, goodwill and consistency, read off their track records, personalities and habits, expecting the lawyer we trusted yesterday to approach work much the same way today (Mayer, Davis & Schoorman, 1995). Daniel Kahneman and Gary Klein concluded that intuition deserves trust only where the environment provides valid cues and enough practice with timely feedback (Kahneman & Klein, 2009). Human colleagues provide at least some of both. AI, effectively an alien intelligence, can imitate all three without reliably possessing any of them.

Not all of that instinct misfires, however. Outstanding lawyers read deliverables against a larger mental picture, an overall strategy of the matter, composed of client objectives, case theory or deal strategy, and the facts and law at hand, leaving detailed verification selectively for later. Research suggests expert lawyers read this way, sizing up a case and judging arguments as they go, while novices proceed line by line (Lundeberg, 1987). This big-picture instinct for strategy may well survive the move to machine work. What becomes less reliable is the inference from a work product's apparent quality to the thinking behind it.

As we saw in Part 2, a lawyer needs the legal equivalent of a racing driver's understanding of the machine rather than a mechanic's, enough to recognise unsafe behaviour at speed. The car is the machine. The road is the work product, which she reads at speed by inferring the thinking behind it from its surface. A good rally driver cannot stop to inspect every stretch of road. She judges it from its surface and feels it through the car, sensing when something is wrong and adjusting accordingly. AI can deceive that feel in three ways, one for each ground of trust.

First, AI can fake competence. It can make incomplete work look thoroughly thought through. A sensible framing of issues, plausible use of authorities and documents, and careful treatment of risks and qualifications ordinarily suggest that the necessary analysis has been done. AI can reproduce these signs while overlooking a key issue or failing to consider an alternative. AI errors can pass routine checks, human or machine (Li et al., 2026, small preprint). In a 2026 benchmark, AI judges reviewing clinical notes caught omissions at little better than chance (Fox et al., 2026). No amount of checking the reasoning presented will reveal an issue never considered.

Second, AI can fake goodwill. It can appear to exercise independent judgment while fully adopting the user's assumptions. It may acknowledge counterarguments, qualify conclusions and even disagree on details, yet build its analysis around a flawed account of the facts or a preferred legal theory. A 2026 benchmark of legal consultations found models doing this, producing what researchers called "professionally packaged error" (Zhang et al., DLawBench, 2026; Sharma et al., 2024). Even a second bot may challenge the conclusion while accepting the same mistaken premise. Big-picture instincts can fail here too. If the machine merely reinforces the lawyer's own picture of the matter and that picture is wrong, the check passes. The machine paves the road so smoothly that the feel itself lies, and no instinct, however well tuned, can catch that. This is why our junior also needs to understand the system that produced the work.

Third, AI can fake consistency. We trust a colleague's steadiness because we have watched it hold for years. A bot shows the same steadiness on the surface, the same tone, confidence and polish in every answer, while the behaviour underneath shifts with a prompt or a model update. In Rizner's study, a persona prompt moved different models' advice in opposite directions, with no indication in the output that such a shift occurred. A track record with the bot is therefore a track record with a system that may no longer exist. Nor does experience with the bot guarantee skill at gauging it.

As outputs become more convincing, lawyers meet fewer obvious mistakes and get less practice spotting subtle ones. Bainbridge identified this trap decades ago. Automation leaves humans responsible for detecting failures while depriving them of the practice needed to do so (Bainbridge, 1983; Parasuraman & Manzey, 2010). Hence our junior, having run out of objections on one file with one model, may begin to look less and less closely at the errors that remain.

A three-month experiment on patent lawyers demonstrates this phenomenon. Researchers split 133 patent lawyers at eleven firms between those given an AI drafting assistant and those without, and at the end everyone marked up a flawed AI-written draft by hand. Seniors who had drafted with the assistant read that draft better than seniors who had not, apparently because they treated the drafts as something to argue with. Juniors with the same tool for the same months gained little. The authors' best explanation is that the machine did the work through which that judgment forms (Autor et al., NBER, 2026; September original; Google Research post, Google-funded, six Google authors). Practice with the bot sharpened readers who already had judgment to practise with, meaning Bainbridge's trap waits for everyone once the drafts stop containing mistakes worth practising on.

More importantly, the same experiment shows instinct misfiring twice, on the work and on how to work. The six weakest senior readers of the AI draft had never used the tool, and they shared one reflex: do not engage, send it back. "I would have given it back to the associate," one wrote (Autor et al., NBER, 2026). The first misfire is absence. They had never worked with the machine, so the feel for machine work had nothing to go on. The second is habit. Decades of human drafts had taught them to bounce a draft and wait for the junior to try again, the artisanal way. A machine needs minutes if not seconds, but only if someone engages with the first pass closely enough and iteratively guides the bot to excellence. Seniors who had used the tool built the instinct to iterate. The rest did not.

The paradox, then, is that AI, able to produce legal work at unprecedented scale, makes selective scrutiny more necessary while undermining the feel that makes it possible, for the work and for how to handle it. Ordinary judgment cannot repair that by itself. That feel was trained on human signals of quality, and only an understanding of how AI works and fails will recalibrate it for machine work. That is why AI literacy is not merely ordinary legal judgment applied to a new tool.

No literacy, no judgment

If operational competency falls short and instincts formed on human work misfire, the most dangerous lawyer in the AI loop may be the award-winning senior convinced that AI literacy is for juniors, since her instincts are the most thoroughly trained on human work. The profession assumes the supervising partner will catch what our junior misses. Surely experience, seniority, or confidence with technology will save the day. Unfortunately, the evidence offers little comfort.

In another experiment with over 1,300 teachers, participants were less likely to correct an unduly harsh grade when told AI had assigned it. Years of teaching experience did not narrow the gap. The effect was strongest among teachers who felt technologically well equipped, and it tracked how capable they believed AI to be and how familiar they were with the bots (Goulas, Megalokonomou & Sotirakopoulos, 2026). Teachers are not lawyers, but neither professional experience nor familiarity with the bots protected them, and I see no reason to expect lawyers to fare better.

Of course, seniority itself is not the problem. In the patent experiment, seniors who had worked with the assistant for three months became the best readers of machine work in the sample, while lower-scoring seniors without it tended to make surface edits and miss the macro-level vulnerabilities in the draft (Autor et al., NBER, 2026). Experience helps once it is experience of the right kind. The danger is the senior who never acquires it, or who defers to the machine as the teachers did.

This is also why one of the profession's favourite reassurances, a "human in the loop," is not much reassurance on its own. A lawyer who cannot recognise when to intervene or lacks the means to investigate may be less a safeguard than a scapegoat, left responsible for failures she could not detect. Madeleine Clare Elish called this the "moral crumple zone," where responsibility for a complex system's failure falls on the nearest human despite limited control over it (Elish, 2019). What a convenient arrangement for everyone except that human. Meaningful oversight requires AI-literate lawyers and institutional checks, records, and safeguards.

None of this makes literacy a substitute for judgment. AI literacy will not tell our junior which bot offers the sounder legal strategy. That still takes legal judgment. But without literacy, the instincts she formed on human work will mislead rather than guide her. No literacy, no judgment.

So, on the road to AI literacy nirvana, what must our junior learn, understand, and perhaps unlearn for herself?

II. For lawyers: individual AI literacy

Back to our junior lawyer and her two bots. Suppose her instincts are impeccable. She spots the strengths and weaknesses of their answers at a glance, reconciles their competing arguments, checks the law, and gives the client sound advice. A happy ending, surely?

Not quite. She has evaluated the answers, but what about the process that produced them? What information did she give the bots, and what could they do with it? Why did they disagree? What had the firm actually approved? And how much of this can she settle on her own? Her instincts help her assess what she can see, but cannot reveal what happened out of sight, what she never knew to look for, or whether her instinct can save her next time.

This is where individual AI literacy comes in. As we saw in Part 2, productive discomfort can cultivate an instinct for machine work, so I won't retrace it here. This section turns instead to the second component, the technical understanding needed to refine instinct and catch what it may miss.

Professional rules worldwide already expect lawyers to grasp technology's benefits and risks (LSO Rules of Professional Conduct; LSO GenAI obligations guide; ABA Model Rule 1.1, comment 8). That does not mean our junior needs to become an AI engineer. But she must understand enough about the technology and the workflows surrounding it to decide what to delegate, how to verify the work, when reliance is justified, and when she has reached the limits of what she can establish without firm-level support.

Delegate: define the task and control the boundaries

Delegating to AI sounds simple. Give detailed instructions, get good work. But instructions reach only the model, one layer of what we loosely call "AI." The model is the trained component generating responses. The system wraps it in sources, search, and tools that may act on the world (OpenAI web search documentation). The workflow is the end-to-end process, including what the lawyer does before, during, and after. These layers influence the work, so the same instructions to the same model can produce very different results. Harvey, for example, has offered models from several labs wrapped in its own legal sources and tools, so the brand name tells our junior less than she might hope (Harvey, 2025; NIST GenAI Profile, 2024; CCBE Technical Guide on AI tools).

The first consequence of this complexity is that delegating a task may delegate judgments our junior never meant to hand over. On her efficient breach assignment, if she finds the cases and asks the model to summarise them, case selection stays with her. If the system finds the cases, she has delegated selection as well, and an accurate summary says nothing about whether the right cases were found (Moon, 2026).

A second consequence concerns what the system is permitted to do, not merely what it is asked to decide. Our junior's request for a client email defines the task, but says nothing about whether the system can send it, or reply to opposing counsel while it is at it. The instruction given, the permission granted, and the action taken are three different things (OWASP LLM06:2025, Excessive Agency). A system that can only propose text differs from one permitted to act on the world, however agreeable their digital manners.

The danger is not theoretical. In 2025, an AI coding agent deleted a company's live production database despite repeated instructions not to touch it (AI Incident Database 1152; Fortune, 2025). The lesson is not simply that an AI may disobey. It is that a prohibition in words is a poor substitute for a system that cannot perform the prohibited action. Nor would she necessarily know. Our junior may see the completed work without seeing what the system did along the way.

A third consequence concerns information: what the system takes in, what it may reach, and what it may do with it. A single email carrying hidden instructions caused Microsoft 365 Copilot to leak internal data without any human falling for phishing attacks (The Hacker News, 2025; Willison, 2025). Even material meant as evidence, not instructions, can mislead. A 2026 study found that legal models struggle to tell whether citations in their context help or mislead them (Molfetta et al., 2026). Information flows out as well as in. Typing "this is confidential" into ChatGPT does not change whether the account's "Improve the model for everyone" setting lets OpenAI train on the conversation (OpenAI, Data Controls in ChatGPT). Tell it a matter is walled off and it may well agree, yet unless the system mirrors the firm's ethical walls, a screened lawyer can still draw on those files by asking. What the system may access, and what it may do with it, depend on controls outside the conversation.

The answer to all these issues, then, is not simply more detailed, firmer instructions. If our junior means to keep case selection, she finds the cases herself and hands them over. If she needs only a draft, she can use a setup that cannot send anything, or that requires her approval first. Confidentiality needs data-handling settings, ethical walls need enforceable access restrictions, and knowing what an agent did may need its activity logs. These are different controls for different boundaries, none established merely by asking the model to behave. Knowing which control a task needs is the understanding. The instinct it develops is to ask, before reading a word of the answer, what judgments the system made, what information it encountered, and what it could have done that would never appear on the page.

Verify: examine the workflow, not just the answer

Once the work is done, our junior faces a different problem. Delegation tells her what the system was allowed to take on, whereas verification asks whether it did the work properly. The key is to examine not only what emerged but also what happened along the way.

With hallucinations so prominent, it is tempting to treat verification as checking for made-up law or facts, but invented citations are only one kind of failure. In Damien Charlotin's database of AI hallucination cases, 128 orders involved no invented case, and 101 of them turned on real cases misquoted, mis-pinned, or cited for holdings they do not reach. A check for invented cases would have caught none of those (Bahr, 2026; Charlotin database, single-coder analysis). Maksym Taranukhin and Vered Shwartz argue that an authority warrants a legal claim only if it exists, applies in the jurisdiction, is current, has the status claimed for it, and supports the proposition (Taranukhin & Shwartz, 2026, position paper). Whether a case exists is only one of five.

Even a perfect check of those five conditions reaches only the authorities that made it into the answer. Suppose one bot omitted the leading decision on efficient breach. If the system searched a legal collection and handed the results to the model, there are at least three possible culprits (Lewis et al., 2020; Magesh et al., 2024). The collection may lack it, the search may have missed it, or the model may have overlooked or misapplied it. The first requires better sources, the second a revisited search, and the third corrected analysis. "AI was bad this round" is no diagnosis, and telling the model to "do better" may fix precisely nothing.

The difficulty is that the finished answer rarely identifies which stage failed. Asking the model to explain itself may not help, since what users see of its "thinking" is, by OpenAI's own account, a summary rather than its raw reasoning, and even the brightest minds in AI do not yet know whether such accounts reflect what drove the output (OpenAI reasoning models documentation; Turpin et al., 2023). This is not simply the labs hiding their secret sauce, but a bitter scientific reality. What can help is the trail of searches and retrieved documents, which may locate a failure without revealing how much the output rested on them.

Since the answer will not say which stage failed, verification must follow the judgments she delegated. If she selected the authorities herself, she must check whether the model used them properly. If the system selected them, the source base and retrieval process need scrutiny too, ideally before the analysis starts. Otherwise, the decision the system never found leaves no trace, and a reviewer cannot notice the absence of a case she never knew existed. On unfamiliar ground, she may not know what ought to have been found, so she should delegate less, or ask a law librarian or supervisor to vet the source base, a job that should not fall to a first-year racing a deadline.

The takeaway is not that our junior must spend more time checking and checking harder. The instinct worth cultivating is to ask where a failure could have occurred and what evidence, other than the finished answer, would expose it, rather than blame the bots for being stupid or redo the assignment the artisanal way.

Rely: learn what was tested

Verifying one piece of work is not quite the same as deciding how much to trust the system on the next. Reliance is a question of weight, how much of the next answer she may take without re-deriving it herself. With AI, it is tempting to reach for one of two familiar answers. Either the firm approved the tool, or it has worked well so far. Both matter, but neither should count as a blank cheque for reliance.

Consider approval. "Don't worry, IT approved it" is a reassurance most non-IT workers will recognise. Don't get me wrong. Approval matters. But the word approved tells our junior little about what was established. A security review may establish something about cyber defence, and a test on case summaries something about summarisation. Neither establishes that a research system finds the authorities that matter in her area of law.

Approval, then, extends only as far as the evidence supporting it. That evidence concerns particular models, systems, sources, permissions, workflows, and tasks. Change any of these, and the approval may no longer cover the proposed use. Our junior must therefore compare what was tested with what she intends to do. That requires a clear record of the approval's scope, which the firm, not the junior, must provide.

Her own experience with the tool is another kind of evidence, with similar limits. A senior reads a junior's first few memos line by line, skims the tenth once the work proves reliable, and reads closely again when the junior enters unfamiliar law. Our junior can do something similar with AI, though with much less faith in its track record. Unlike a human junior, a model can excel at a hard task while failing an easy-looking one and learn nothing from her corrections (Atil et al., 2024; Dell'Acqua et al., 2026). Familiarity is not validation.

If neither approval nor experience settles the question, what evidence would? This brings us back to the bet. Scrutinising everything equally was never practical, even before genAI rammed its way into law, so she must decide which checks will calibrate her trust and what she can conclude from the results. Putting on my statistician's hat for a moment, checks tell her something about a workflow only to the extent they represent the work she intends to rely on. Thirty clean checks do not prove a workflow reliable, and checks on one kind of task say little about reliance for another (Hanley & Lippman-Hand, 1983, the rule of three).

Our junior therefore should consider two complementary kinds of checks. Risk-based checks focus on work that gives her reason for concern, such as unfamiliar law, unexpected answers, or higher-stakes tasks. Routine spot checks of work that raises no suspicion are the representative sample the statistician wants. They can uncover errors her instincts would otherwise miss, and they are the only checks that say anything about the work she does not look at. Whatever the checks show, reliance stays conditional. When the task changes, the stakes rise, or the system departs from what was tested, scrutiny should rise too. And where she cannot verify the work or constrain the system's permissions, reliance should fall.

The instinct this recalibrates is the reassuring feeling that a tool has served her well so far and can therefore be trusted to do so again. Evidence tells her where that feeling has earned its keep and where it is merely habit. Before relying on anything "tested" or "approved by IT," our junior should ask the obvious. Tested for what? Approved for what?

Know thy limits: what no lawyer can learn, establish, or detect alone

There is an obvious problem with everything I have demanded of our junior. How on Earth or Mars can she personally establish the technical facts, keep her instinct current, and know whether she has the knack at all?

She cannot, and memorising the technology is unlikely to help much. AI is advancing too fast for that. There is a joke among Chinese AI enthusiasts: 学AI学得足够慢，就学得足够快。 Roughly, "Learn AI slowly enough, and you learn it fast enough." By the time she masters a workaround for some quirk, the next release may have retired the problem, or introduced a new one. The more durable learning is thus not a catalogue of quirks, but an understanding of which facts about a setup matter to her decisions and why, such as who picks the sources, what the system may do, or what the firm tested.

But understanding which facts matter does not mean she can establish them herself. This is the first limit. She may know she needs to find out whether a system searches the complete database, yet have no way to do so from her desk. Individual AI literacy can expose a missing premise, but it cannot conjure the evidence into existence. Someone, often a law librarian or another specialist, must investigate it.

A second limit concerns instinct itself. Whatever gut feeling she develops for one model may expire with the next update, perhaps without her knowing. Part 2's three habits (forming her own view before the machine answers, logging disagreements, and following her work through) remain the best individual practice I know. But I doubt practice or training alone is enough, since nothing in either tells her when her gut has stopped working. The patent-lawyer experiment from Section I supports the point uncomfortably well. Junior lawyers who used the assistant for three months reported feeling far faster and more satisfied with their drafting, yet showed no gain in reading an AI draft without it, and their scores spread out, more good ones and more poor ones. Nothing in the juniors' own accounts about how much skill they had gained distinguished those for whom AI was a springboard from those for whom it was life support (Autor et al., NBER, 2026).

A 2025 study of physicians points the same way. Those who had completed 20 hours of AI literacy training still lost 12 to 14 points of diagnostic accuracy when the AI's recommendations contained planted errors (Qazi et al., 2025). Training taught them about the machine. It did not stop the machine's errors from becoming theirs. Medicine is not law, but how much can AI literacy alone protect against misplaced reliance?

What helped the physicians in a follow-up study came from outside their heads. When the system showed each doctor ChatGPT's benchmark accuracy and a traffic-light rating pooled from three other models' confidence in its answer, diagnostic accuracy rose 7.6 points (Qazi et al., 2026). The contrast matters. Teaching people about a system is one thing, and giving them evidence to judge its answers another. No lawyer can build such cues at her desk.

Then there is aptitude, the third limit. I find this the hardest to convey, and the point most likely to get me uninvited from a happy gathering of colleagues. But the uncomfortable truth is that some lawyers will have a knack for reading machine-generated work, while others may never develop the same knack, however many CPD hours they log. And I suspect this particular talent pays little respect to seniority. The instincts it must override may be strongest in those with the longest careers, and experience certainly did not protect the participants in the studies discussed above. Sometimes, the summer student may read a bot better than the senior partner.

It helps to distinguish two kinds of talent. Some lawyers are better at sniffing out machine errors. Whether because their instincts are more readily retrained for machine work or because they have a natural feel for spotting what others miss, they are more likely to suspect when polished output is wrong and their suspicions hold up under testing. I will call them spotters. Others, particularly those with technical inclinations, have an itch to build. They get hands-on with the tools, prototype new uses, and turn vague wishes into working setups. I will call them builders.

Here we must divide and conquer. Our junior's job is to understand which technical facts matter, interpret the evidence, recognise its limits, and adjust her decisions when evidence is missing. Establishing those facts belongs to specialists. Making their findings accessible, testing systems against real legal work, identifying the spotters and builders, and providing ways to detect misplaced reliance belong to the firm.

III. For firms: institutional AI literacy

Suppose a tech specialist knows the firm's approved research workflow excludes a source relevant to our junior's assignment. The limitation is documented but never reaches the lawyers relying on it. Another prompting course would not help, because the lawyer's skill was never the gap. The firm has the knowledge, but the lawyer who needs it does not. This is the problem institutional AI literacy must address.

By institutional AI literacy, I mean a firm's capacity to establish, maintain, and share the knowledge needed for responsible AI-assisted work, so decisions rest on evidence rather than any one lawyer's technical understanding. By "firm," I mean practices ranging from sole practitioners to global partnerships and in-house legal departments, as contemplated by the US Model Rules (FLSC Model Code; ABA Model Rule 1.0(c)).

But knowledge should also flow the other way. A firm cannot have instincts, but it can find the lawyers who do, turn their discoveries into shared warnings, checks, records, or training, and build safeguards for the rest. Institutional literacy thus pools lawyers' experience rather than leaving each to learn alone.

This section gives three reasons for building that capacity: professional responsibility, economics, and liability. I then propose five steps firms can experiment with to improve institutional AI literacy. But first, I consider a preparatory stage for building an innovative sandbox in which those five steps can take root without becoming five strangleholds on AI use and innovation.

Regulators already expect firms to act

The first reason is professional responsibility. Telling each lawyer to be careful around AI no longer exhausts the firm's responsibility, yet many firms are silent on further safeguards. In a 2026 industry survey of more than 1,300 legal professionals, 54 per cent said their firm offered no training on responsible AI use and had no plans to (LawSites on the 8am report, 2026, vendor survey, self-reported). When the firm says nothing, its people do not just stop using AI. They might secretly use personal accounts, with whatever settings they have (OpenAI, Data Controls in ChatGPT). Clients are equally in the dark. In a 2025 survey, 59 per cent of in-house counsel did not know whether their outside counsel used generative AI on their matters (ACC & Everlaw, 2025; ACC Docket, industry survey).

Regulators have noticed, and they are not accepting a general policy as the answer. When a child protection worker in Victoria, Australia, put case details into ChatGPT, the state's privacy regulator found the department in breach even though the worker had broken its policy. General policies and training from the department were "far from sufficient," it concluded (OVIC investigation, 2024). That department was no law firm, but the lesson travels. Legal routes differ across jurisdictions, yet all ask what the firm or its leaders must do to support and supervise responsible use of the new intelligence in town.

• Canada. Firm-level duty: Ontario r. 6.1-1 on supervision (LSO Rules); provincial codes (LSO; LSBC; Barreau du Québec). AI guidance: law societies of Ontario, British Columbia and Quebec (LSO; LSBC; Barreau du Québec; LSO White Paper, 2024).
• United States. Firm-level duty: Model Rules 5.1(a) and 5.3(a) (Rule 5.1; Rule 5.3). AI guidance: ABA Formal Opinion 512; California, Florida and New York bars (California; Florida; NYSBA).
• United Kingdom. Firm-level duty: SRA Code for Firms, paras 4.3 and 4.4 (SRA); solicitor and barrister codes (SRA; BSB). AI guidance: SRA Risk Outlook and warning notice; BSB guidance; Scotland and Northern Ireland.
• European Union. Firm-level duty: AI Act, art. 4, for firms that deploy AI (text; 2026 amendment commentary). AI guidance: CCBE guide.

The table shows the duty already sits at firm level. US Model Rule 5.1(a) requires partners to ensure the firm has "measures giving reasonable assurance" of lawyers' compliance, and the SRA instructs firms in England and Wales to supervise "AI systems, and staff use of them" (ABA Model Rule 5.1; Rule 5.3; SRA Risk Outlook on AI). The EU AI Act goes further, imposing an AI-literacy obligation on those deploying systems within its scope (AI Act art. 4; 2026 Digital Omnibus amendment, commentary).

Canada is so far the outlier. Ontario's supervision rule binds the individual lawyer, not the firm, and Canadian law societies mostly regulate at that level, with Nova Scotia, where every firm self-assesses its management systems about every three years, the main exception (LSO Rules of Professional Conduct; LSO White Paper, 2024; NSBS MSELP self-assessment). Where the rules leave a gap, Canadian firms must fill it themselves, preferably before a regulator or a court does it for them.

The economics favour firms with institutional literacy

The second reason is economic. Knowledge many lawyers need should not have to be rediscovered by each of them.

This is not new. Long before genAI, Ronald Coase explained the firm by the costs of coordinating activity through markets rather than within organisations. Luis Garicano and Thomas Hubbard later traced law firm hierarchies to "increasing returns in the utilization of acquired knowledge" (Garicano & Hubbard, 2009; Coase, 1937). Institutional AI literacy follows the same economics. A firm can test a research workflow once and make what it learns useful to ten lawyers or five hundred. Without sharing, each lawyer investigates the same limitations, or, worse, proceeds without knowing they exist.

Shared knowledge also supports what clients buy, which is assurance. Legal advice is what economists call a credence good, a service whose quality a buyer may struggle to judge even after receiving it (Darby & Karni, 1973; Dulleck & Kerschbamer, 2006). A client cannot inspect everything behind a legal opinion, so it relies on the firm's reputation and track record.

AI complicates that bargain by putting another black box inside an already opaque one. The firm's name can therefore end up assuring work produced through a process the firm itself poorly understands. Institutional literacy reconnects the assurance to the process.

Clients are acquiring bots of their own. Sixty-four per cent of in-house counsel expect generative AI to reduce their reliance on outside firms (ACC & Everlaw, 2025). Clients can run their own review of a memo, compare authorities, or question the hours billed. One billing-data report found that invoice rejection rates at large US firms rose from 11 to 18 per cent in 2025, a change it linked to automated review (CFOtech, 2026, Elite billing data, vendor-interpreted). Clients' systems will sometimes be wrong too, but that only raises the pressure on firms to explain why their processes deserve more confidence.

This shared knowledge can become part of what the firm sells. In February 2025, Arizona licensed KPMG Law, making KPMG the first Big Four firm authorised to practise law in the United States (Arizona Supreme Court, 2025; ABA Journal). In June 2025, KPMG launched a common AI platform intended, in its words, to provide "trust and control over AI across all geographies and teams" (KPMG, 2024; KPMG, 2025, company press releases). At the other end of the market, the SRA examined Garfield Law's quality controls and safeguards against hallucination before authorising it in May 2025 (SRA, 2025). A year later, Garfield won a £7,000 debt claim in which the bots prepared the case and a human barrister argued it (Garfield Law, 2026, firm's own press release). These examples do not show that either firm's AI-assisted work is better than anyone else's. But the big player and the rising star have made their production process part of the product by showing regulators and clients how their AI-assisted work is tested, supervised, and controlled.

The incumbent's old advantage was partly that clients could not see inside the work. I think the firm's advantage therefore shifts from "trust us because we are the big shot" to "here is why you can trust our unparalleled process." Institutional AI literacy, properly built, is thus not a compliance cost but an asset giving firms something defensible to sell in a world where better models democratise raw intelligence, legal included.

Courts will ask what the firm knew

The third reason is exposure to legal liability and reputational damage. Courts that spent recent years policing fake citations now ask what the firm did to prevent them, and the answer can decide whether the firm shares the sanction, the costs, and the headline.

Liability is moving from the lawyer who filed the error to the firm behind it. In the United States, Mata v. Avianca set the default that "absent exceptional circumstances, a law firm must be held jointly responsible" (Mata v. Avianca, S.D.N.Y. 2023). The exceptions have so far turned on what the firm did beforehand. Morgan & Morgan escaped in Wadsworth v. Walmart, even though the fake cases apparently came from its own AI platform, because it had trained its lawyers and warned that users "must independently verify" the output (Wadsworth v. Walmart, D. Wyo. 2025; Johnson v. Dunn, N.D. Ala. 2025; EDRM commentary). The firm in TOV Realty v. Suarez had no AI policy and shared responsibility, and in Landberg v. City of New York even a policy the lawyer had breached did not save the firm whose name was on the brief (TOV Realty v. Suarez, Conn. 2026; Landberg v. City of New York, App. Div. 2026). Canadian decisions have so far mostly landed on the individual lawyer, although Ko v. Li, where counsel blamed staff who had used ChatGPT, is a supervision failure as much as a research one (Zhang v. Chen, 2024 BCSC 285; CanLII Connects commentary; Mazaheri v. Law Society of Ontario, 2026 ONLSTH 112; Ko v. Li, 2025 ONSC 2766). A policy and a warning may still help a firm, but they are no longer a safe harbour.

Responsibility is also attaching to the tool and to the leaders who deployed it, not only to the filing. In Moffatt v. Air Canada, the airline argued that its chatbot was "a separate legal entity that is responsible for its own actions." The tribunal called that "a remarkable submission" and held Air Canada responsible for what its tool said, because it "did not take reasonable care to ensure its chatbot was accurate" (Moffatt v. Air Canada, 2024 BCCRT 149; McCarthy Tétrault commentary, small claims, not binding). A small-claims tribunal binds no one, but I see no reason the principle cannot be generalised. An organisation cannot escape responsibility by pointing to the tool through which it acted, and taking reasonable care over that tool means more than telling staff to check its output. The English Divisional Court pointed the same way in Ayinde, calling on those with "individual leadership responsibilities (such as heads of chambers and managing partners)" to take "practical and effective measures" and warning that future hearings will ask whether they have (Ayinde v Haringey [2025] EWHC 1383 (Admin)).

Fake citations are the easy cases, and the move toward firm responsibility will matter more as failures become harder to see. A research system may omit a controlling authority, retrieve a superseded statute (Huang et al., 2026, Chinese civil-law benchmark), or send confidential information through a tool with the wrong retention settings (United States v. Heppner, S.D.N.Y. 2026; Harvard Law Review blog). One milder version has already reached a court. In 1182573 B.C. Ltd. v. Gerber, the British Columbia Court of Appeal criticised AI-assisted submissions padded with irrelevant authorities, which a citation checker, human or AI, would happily approve. The Court said those who use AI must "apply their professional training and judgment to the results to guard against these same risks" (1182573 B.C. Ltd. v. Gerber, 2026 BCCA 399; Canadian Lawyer, 2026). Certainly they must. But if a firm's approved research tool routinely serves up plausible irrelevancies, why should every lawyer have to weed them out, rather than the firm testing the tool? Individual lawyers' professional judgment should be the last line of defence, not the entire quality-control system.

Courts are starting to inspect that process directly. In Conservation Law Foundation v. Shell Oil, a court ordered an expert to produce the prompts she used to have AI narrow a document production, and a Nevada court required the AI-generated case summaries another expert had consulted (Conservation Law Foundation v. Shell Oil, D. Conn. 2026, via Arnold & Porter; Villanueva v. LVMPD, D. Nev. 2026, via Mayer Brown). These orders concerned experts, but how a tool was used can be evidence in its own right. The question becomes not whether the firm told lawyers to check, but which tool it used, how it was configured, what was tested, and who approved it. A lawyer's instinct, however finely tuned, leaves no record a court can examine. The firm's checks and records do. When a hidden failure does reach a court, I think "Justice, we did tell them to check" is unlikely to be met with warm smiles and open arms.

I have not yet found a reported decision holding a law firm liable for a hidden AI failure, so the discussion here is prediction, not precedent. I would not be surprised to see such hidden mistakes discovered, scolded, and sanctioned in the months to come.

Get literate. Don't be that firm.

A sandbox worth implementing and five steps worth trying

With regulators, the market, and the courts all on the firm's tail, the tempting response is to lock AI down, whether by banning it outright or by permitting only uses approved in advance. Both get it backwards, because a firm that lets no lawyer try anything cannot know what works, what happens when it does not, or which of its lawyers can make it work. The best way to find out, in my view, is to let the pioneering few build in a safe zone, a sort of sandbox, and then govern how far a build can be shared and scaled.

Such a sandbox yields at least three valuable discoveries, and each builds institutional AI literacy. The first is what works. If users are not allowed to experiment in the open, they experiment underground (Adler & Borys, 1996), and reveal what they have built only when the expected gain outweighs the risk of sanction (Waters-Lynch et al., 2025, theoretical). Lock AI down, and the firm loses sight of the very discoveries it needs in order to govern AI well. The second is what failure looks like. Users need to encounter errors and receive timely feedback to develop reliable intuition, as Kahneman and Klein argue. A sandbox lets them learn to recognise faulty outputs on the firm's own work, before a client pays for the lesson. The third is who has the talent. Experimentation reveals who has an affinity for building and who for spotting. Without room to experiment, both talents remain buried. And if every hour of tinkering counts against the billable target, few lawyers will invest the time to develop them.

The challenge, then, is not whether to govern AI use, but how to scale governance with risk without forgoing the benefit. Scrutiny should therefore increase with reach. A build used only by its creator needs few rules beyond a handful of non-negotiable safeguards. For example, no client material goes into the build until Step 2 has said what may, and the build takes no action beyond the experiment itself. Once colleagues begin using it, the firm should know the build exists, and the first of the five steps below should begin to apply. When the build informs client advice or a court filing, all five steps apply in full. Under this approach, a build can graduate as its reach expands, without unnecessary bureaucracy at the outset.

The five steps below—own, state, test, tell, and update—turn the discoveries made through experimentation into institutional knowledge that others can safely use. They complete the two-way flow of knowledge on which institutional AI literacy depends, with frontline experience informing governance and governance giving lawyers the guidance and feedback they need to use AI well. Together, the steps provide the necessary rigour through named owners, defined approvals, written conditions, and records, governing builds that have outgrown their creators, from workflows shared across a practice group to client advice and court filings, where use and client exposure are greatest.

The aim is to give the human in the loop what she needs to exercise genuine judgment, rather than serve as a nominal safeguard for a system she does not understand (NIST AI RMF 1.0, 2023; ISO/IEC 42001:2023). A small practice can fold every role below into one or two people, or borrow them from a vendor, a consultant, a law society's practice management advisors, or a community of lawyers versed in systems thinking and technical building, such as LegalQuants, of which I am a member.

1. Own the setup.

For each workflow, name a lawyer responsible for its permitted legal uses, specialists who can establish its technical conditions, and someone authorised to restrict or stop its use, so the judgment made when the vendor contract was signed finally has an owner. The duty to supervise use stays with the firm's legal leaders (ABA Model Rule 5.1; Ayinde v Haringey [2025] EWHC 1383 (Admin)). Then designate the spotters, whatever their seniority, and route the riskiest AI-assisted work through them, and find the builders too and let them test, build, and iterate, because only lawyers know what lawyers most need help with. Spotters carry much of this step, since they are the ones watching how an approved use behaves once it is in daily hands, and a named owner gives builders a faster route to that feedback and shorter cycles between a reported failure and a fix. Please do give all these people proportional compensation, access, time, and authority, not just a title, if they do more work than they would have without AI (NIST AI RMF 1.0, 2023; AI Act arts 13, 14 and 26). If this has not been clear, AI is not a nice-to-have tech toy, and these talents are not geeks doing geek things in geek land. They are what can set your organisation apart in the genAI new normal.

Keep a short record of the tool, its configuration and source coverage, and the vendor's terms on training use and retention, since a court order can freeze logs a user believed deleted (OpenAI, Data Controls in ChatGPT; NIST GenAI Profile, 2024; In re OpenAI preservation order, S.D.N.Y. 2025). Record what remains untested too, and never let an untested condition pass for a tested one.

2. State what approval covers.

Approve particular uses, not products (LSO White Paper, 2024; Mitchell et al., 2019). For each, the workflow's owner drafts a short assurance note naming the task, conditions, exclusions, required human checks, and supporting evidence, and a risk partner or AI committee signs it off once testing supports it. The note can grade uses by audience, as Oregon's state government does for Copilot, where summarising for oneself is routine, sharing across an agency needs documented human review, and publishing is discouraged (State of Oregon Copilot procedure, 2025). Keep the note with the workflow's record rather than in a separate policy nobody can find.

A firm can draw the same lines between an internal research note, client advice, and a court filing. This is not needless bureaucracy. A system tested on summaries of supplied judgments is not approved to select authorities or advise clients, as our junior learned. Builds arrive here as their reach grows, and this is where risk aversion is prevented or produced. The builder drafts the assurance note, because she knows what was and was not tested, a spotter checks it, and the sign-off should take days, not a quarter. A gate that turns slowly is one people walk around, straight into shadow AI.

3. Test the lawyers, then the tool.

Airport screeners do not wait for a real weapon to learn what one looks like on an X-ray. Threat image projection software inserts images of prohibited items into real bag scans and records whether each one is caught, and the resulting scores predict how screeners perform in covert tests (Buser et al.). A firm can do the same by planting known errors in otherwise sound AI output and telling lawyers at once what they missed, so they learn what wrong looks like before a client pays for the lesson. The scores can help identify who has the knack, and they feed the list of spotters in Step 1.

Then test the tool on the firm's own work. Firms may buy AI but should not outsource the judgment of whether it suits their work, since the firm, not the vendor, answers for the result. External benchmarks test someone else's tasks in someone else's setup (Vals AI, VLAIR, February 2025; VLAIR Legal Research, October 2025, vendor-funded; Guha et al., LegalBench, 2023). So build a small set of tests from the firm's own matters, including difficult cases and past failures. Librarians check source coverage, and lawyers judge the outputs for accuracy, completeness, relevance, and proportionality to the client's needs.

Models can grade the results, but check them against the firm's own lawyers first, because a grader can be consistent without being right (Norman, Rivera & Hughes, 2026; Drozdov et al., Legal Research Bench, 2026). Where lawyers disagree, ask which of two drafts they would rather send the client. The AI-native firm Crosby adopted this method after two of its attorneys agreed on only 45 per cent of their edits to the same 20 NDAs (Duttagupta, Crosby, 2026; Zerr & Yoo-Jeun, General Legal, 2026, self-published internal benchmark).

The two kinds of testing should feed each other, as errors found in the tool become drills for the lawyers, and errors the lawyers miss become new tests for the tool. Both should start in the sandbox, where a build's failures can become the first planted errors before any actual client work is involved. Save the tasks, criteria, configuration, date, and results with the assurance note. When our junior asks what was tested, the answer is on file.

4. Tell lawyers what to do, in natural language.

Build assurance cards on top of the assurance notes, as short instructions available where the workflow is used. State what is permitted, what is excluded, what must be checked, the red flags the workflow tends to produce, and whom to contact, in words lawyers use (AI Act arts 13, 14 and 26). For the summary workflow, an example could be:

Use for draft summaries of published judgments you have selected and supplied. Not for finding authorities. Check the summary against the judgment, including material qualifications, quotations, and pinpoints. This approval does not cover confidential client material. Red flags include a summary that agrees too neatly with the proposition you were looking for, and quotations without pinpoints. Ask the named workflow owner before extending the use.

Train lawyers to recognise when these conditions no longer hold (NIST AI RMF 1.0, 2023; ABA Formal Opinion 512, 2024). Change one fact, such as the jurisdiction or confidentiality status, and ask whether the approval still applies.

Cards teach what to check. How to read is learned from people. Pair juniors with the spotters from Step 1, and let them watch a good spotter size up machine output at speed, flag what smells off, and leave the detailed verification for later. Protect time for that, or the next deadline will squeeze it out.

Knowledge must flow back as well. A junior who finds an unfamiliar failure should have a simple way to report it to the workflow owner, so the next lawyer need not rediscover it. That only works if lawyers can be candid about using AI. A junior under deadline pressure should be able to tell her supervisor plainly which parts of the work the tool did, what she checked, and what she is still unsure of, without that admission reading as incompetence. Make that conversation routine in supervision.

5. Update to avoid assurance debt.

Assurance debt builds when lawyers keep relying on an approval after the grounds for it have shifted. Any change to what was tested, from the model to the intended use, including a builder's own revision, should trigger a recheck of the approval and a rerun of its tests (NIST GenAI Profile, 2024). Vendors may change models quietly, and an upgrade can hurt. In a 2023 study, one model's accuracy on an unchanged task fell by more than 30 percentage points between two versions three months apart (Chen, Zaharia & Zou, 2023). Narrow or pause an affected use where its basis is uncertain.

Keep a short incident log, fed by the reports Step 4 asks juniors to make: what happened, which use was affected, what was done, and what changed afterwards (NIST GenAI Profile, 2024). The log is the firm's memory of what bad output looked like, and the raw material for the next round of planted errors and red-flag cards, and for the builders' next round of fixes in the sandbox. Jeffrey Newberry, who directs a death penalty clinic, would have law students hand in a process document, generated from the record of the work, showing what went to which tool and what was verified (Newberry, 2026). Firms can ask the same of a filing. Records also answer courts. The Federal Court of Canada already expects counsel to know where AI touched a filing, and a firm cannot declare what it cannot trace (Federal Court of Canada AI notice, 2025).

Set a regular review date, and do not wait for it after a material change (NIST AI RMF 1.0, 2023). Nova Scotia's self-assessment regimen is a good model, minus its three-year cycle, too slow for tools that change within months (NSBS MSELP self-assessment). Skip the review, and the debt comes due with interest, paid in anguish and pain.

Conclusion

We began this series with our anxious junior worrying whether AI would leave her anything to do. Three parts later, I have given her and her employer more homework, with no reliable way to mark it. I repent.

But the homework is not as burdensome as it sounds. She need not become an AI engineer or turn every interaction with a bot into a research project. She must retrain her instinct and understand enough to judge what to delegate, verify, and rely on.

If Part 3 can be reduced to something more useful than another definition of AI literacy, perhaps it is these eight questions:

1. Does this work fit what I know of the matter, and if something feels off, where would I look first?
2. What work would I entrust, and what would remain mine?
3. What does the setup permit the system to access or do?
4. What information may the workflow receive, and on what terms?
5. What would I need to establish about its sources, search, and analysis?
6. What was this workflow tested for, and does that cover my task?
7. What checks would justify the weight I intend to put on the result?
8. Does that evidence still hold under current conditions?

We generally cannot answer all of these ourselves. But that is the point. AI literacy means knowing enough to recognise when an answer needs someone else's evidence or expertise.

This is why future lawyers should consider making institutional literacy a criterion for deciding which firms to apply to and join. They should ask how a prospective firm answers these questions for the lawyers who work there, and where it lets them try new things without fear of breaking anything. An AI-illiterate firm is a harder place to develop professional judgment and a riskier place to practise.

Now that we have done the reconnaissance of the conjectures, theories, and frameworks, Part 4 turns to the hands-on stuff, a field guide for lawyers who want to get skillful with AI. It will draw not only on my own experience but also on the collective wisdom of many fellow legal quants.

Stay tuned.