Tag Archives: LLM

Fifteen Years in Your Court

21 Friday Aug 2026

Posted by in ai, Computer Forensics, E-Discovery, General Technology Posts, Law Practice & Procedure, Personal, Uncategorized

≈ 3 Comments

Tags

, , , , , , , , , , , , ,

Fifteen years ago yesterday, frustrated that a legal-media conglomerate had put a decade of my articles behind a paywall, I started Ball in Your Court. From that christening to this quinceañera, I’ve penned 283 posts viewed 690,650 times. More than 2,000 people subscribe to receive them. I’m grateful to everyone who tolerates the updates and indebted to the roughly half who take the time to click through.

The numbers are gratifying, but the real reward has been hearing from readers—especially when you correct me, challenge me or tell me that something here helped you do the work better. I started the blog to share what I had learned; fifteen years later, I’ve learned at least as much from those who read it.

For reasons I can’t fathom, more than 6,000 people read my post of August 28, 2024 on the day it appeared. The key takeaway from that short post about linked attachments was: “We should never guess at what we can readily measure.”

For reasons I fully fathom, my most visited post was one published a year ago this week, marking the death of EDRM’s Kaylee Walstad. It has been a year of missing her every day. If you knew Kaylee, please pause a moment to remember her.

I christened this blog thusly:

The exabytes of digital information streaming about us today are rich rivers of evidence that will help us find the truth and move us to do justice more swiftly, more economically and more honorably than ever before. It will require every litigator to master new skills and tools, and alter the approaches and attitudes we bring to the adversarial process. We must reinvent ourselves to master modern evidence or be content with a justice system that best serves the well-heeled and the corrupt. The path to justice is paved with competent evidence and trod by counsel competent in its use.

I wrote that years before AI and deepfakes entered the conversation. It is truer now, and its mandate more compelling, when the justice system—and the Department of Justice—is deeply in thrall to the well-heeled and the corrupt.

AI can amplify competence, but it also allows anyone to feign it. Now, more than ever, we must continually reinvent ourselves to master modern evidence and be prepared to expose the fake and the fabulous.

Drafting RFPs for Robots to Read

20 Monday Jul 2026

Posted by in ai, E-Discovery, Law Practice & Procedure

≈ 2 Comments

Tags

, , , , , , , , , , ,

Time flies: Two years ago, in a post here, I floated the proposition that if the other side is going to hand your requests for production to a large language model and let the machine decide what’s responsive, then let’s draft those requests with the machine in mind. Doug Austin was generous enough to amplify the idea a week later. Two years on, it’s no longer something to merely think about; producing parties are ceding first-pass relevance review to LLMs.  The request in front of the model is now a prompt whether we know it or not.

So, let’s make that prompt our own.

A large language model doesn’t share the experience a seasoned reviewer brings to “all documents touching or concerning the transaction.” Ambiguity that a human reviewer resolves by instinct becomes, for a model, a coin flip between over- and under-inclusion. If you want their AI to find what you need, then tell it how to discriminate.

Seen that way, an AI-aware request for production is doing one of two things. At a minimum, it lards the request with enough discrete elements—custodians, systems, defined terms, date ranges, document types, examples—that opposing counsel can hardly avoid building those elements into whatever prompt they feed their review platform. At best (insofar as the rules of procedure permit or human sloth promotes), it hands them a fully formed prompt: language so ready to roll that the path of least resistance is to paste it straight away. The first mode constrains by specificity; the second exploits the happy truth that a good prompt ready-made is a prompt somebody will be tempted to deploy. Either way, the benefits are the same: clarity over boilerplate, context over conclusion, defined terms over loose keywords, and illustrative examples that let the model pattern-match to the documents you need.

None of this is way out there. It’s just good drafting, made newly consequential because the first reader is now a machine.

Oh, Those Pesky Rules

The Federal Rules reward this technique. FRCP Rule 34(b)(1)(A) requires that a request “describe with reasonable particularity each item or category of items to be inspected.” Particularity and prompt-craft pull on the same oar: both esteem the concrete over the conclusory. An AI-aware request isn’t a departure from Rule 34; it’s Rule 34 taken seriously by a lawyer who recognizes a model is on the other end.

Cautions to Stay Inside the Guardrails

First, drafting a request is not the same as running the other side’s review.  The Sedona Principles, Third Edition, Principle 6, holds that “responding parties are best situated to evaluate the procedures, methodologies, and technologies appropriate for preserving and producing their own electronically stored information.” I’ve quarreled with Sedona Six before—competence is something a producing party should endeavor to earn, not something we should presume—but the principle still rears its ugly head to shield the tools the other side chooses. So, the line to walk is this: an AI-aware request steers relevance and particularity; it does not dictate the responding party’s platform. Write the request to tell the model, any model, what responsiveness looks like. Don’t write it so as to effectively tell opposing counsel which model to choose or how to configure it. The former is advocacy and fair game. The latter invites a well-founded objection.

Second, particularity is a gun that kicks as hard as it shoots. The more precisely we enumerate document types and search terms, the greater the risk that a producing party treats a careful list as the outer boundary of the request and withholds everything beyond it. Two years ago, I noted this language would be “unlikely to be embraced by counsel ever-apprehensive of framing a request too-narrowly,” and the worry remains. The fix is craftsmanship: pair concrete guidance with a stated purpose and, okay, keep your cherished “including but not limited to,” so your examples instruct the model without unduly shrinking the scope.

There’s a third point worth mentioning, because it proves prompts are becoming discovery objects in their own right. In Conservation Law Foundation, Inc. v. Shell Oil Co., No. 3:21-cv-00933 (D. Conn. May 18, 2026), a magistrate judge ordered production of the prompts an expert used to drive an AI tool, treating them as fair game for discovery into methodology. That case concerns an expert’s prompts, not the language of an RFP (and, frankly, I don’t think the judge got it right in the face of a stipulation between the parties); so don’t read too much into it.  Still, the decision signals that courts have started to regard AI prompts as part of the discovery record. The prompt-craft we bring to our requests and the prompts our adversaries feed their review platforms are drifting toward daylight.

How Does It Work?

Take a matter everyone remembers. In the Dominion Voting Systems’ defamation suit against Fox News—the case that settled for $787.5 million in April 2023—the fight turned on what people inside Fox knew about the falsity of the fraud claims their network kept airing. A conventional, human-oriented request in that case might read like this, and requests like it are served every day:

All Documents and Communications relating to Dominion Voting Systems, including any allegation of fraud, vote manipulation, algorithmic “vote switching,” or foreign influence involving Dominion’s products in connection with the November 2020 U.S. presidential election.

Does a senior associate knows what to do with that? An AI model told to sort a Fox custodian’s mailbox against it will drown—everything “relates to” Dominion in a case about Dominion! A fallback to keywords will be, at once, over- and under-inclusive.

Let’s rewrite the request as something a reviewer will be sorely tempted to paste straight into a review tool:

Identify and produce every Document and Communication—including emails, text and Signal messages, Slack messages, on-air scripts, booking notes, and drafts—in which any Fox News host, producer, booker, or executive discussed, doubted, questioned, promoted, or sought to substantiate the claim that Dominion Voting Systems’ machines or software switched, deleted, or altered votes, or were connected to Smartmatic, Venezuela, Hugo Chávez, or the “Kraken.” The purpose of this request is to surface each custodian’s internal knowledge of the truth or falsity of the on-air fraud allegations. Responsive material will typically originate with custodians including the hosts and executives identified in Schedule A, date from November 1, 2020 through the network’s 2021 on-air corrections, and use terms such as “Dominion,” “Powell,” “Giuliani,” “rigged,” “switch,” “Smartmatic,” “Chávez,” or “crazy.” An example of a responsive document is an internal text message in which a host privately derided the fraud claims as baseless while the network continued to air them.

Notice what the second version does and doesn’t do. It gives the model purpose, custodians, boundaries, defined terms, and an example—everything a prompt needs, and enough discrete hooks that the other side can’t build its own prompt without importing most of them. It says nothing about which review platform to buy or how to configure and train it.

Use the Tools

One last point: If we’re going to draft requests for a machine to read, we might as well let a machine do the heavy lifting. Today’s subscription-tier models—the paid versions of ChatGPT, Claude, Gemini, or ChatGPT, not the free tiers coasting on last year’s tech—handle this conversion with ease. Feed one your draft and the bare contours of the case and let it do the heavy lifting. The prompt need be no fancier than this:

You are an experienced litigator revising a request for production so an AI tool conducting first-pass relevance review will read it accurately. Rewrite the request below to add reasonable particularity: name the likely custodians and data sources, add a one-sentence statement of purpose, enumerate the pertinent document types, list the defined terms and search language, set a sensible date range, and give one example of a responsive document—without dictating the responding party’s review methodology or narrowing the request’s reach. Here’s the request: [paste].

As always, mind the confidentiality of whatever you paste, use an account that won’t train on your inputs, and read every word the machine spits back—but let it spare you the first draft.

The robots are reading our requests now. Shouldn’t we write for our real audience?

2026 Guide to AI and LLMs in Trial Practice

09 Friday Jan 2026

Posted by in Uncategorized

≈ 2 Comments

Tags

, , , , , , ,

It’s been one year today since I published my introductory primer called Practical Uses for AI and LLMs in Trial Practice. AI changes so rapidly, I’ve been burning the midnight oil to overhaul and expand the work, now entitled Leery Lawyer’s Guide to AI and LLMs in Trial Practice. It’s no mere face lift, but a from-the-ground-up rewrite reflecting how AI and large language models power trial lawyer tasks today. Since the first edition, AI has moved from curiosity to necessity. Tools like ChatGPT and Harvey are no longer novelties, and the economics of AI-assisted drafting, discovery management, and record comprehension are undeniable. At the same time, the risks of use are better understood. Hallucinations, overreach, privilege exposure, and misplaced confidence are genuine, and the guide meets them head-on, offering practical guardrails and practice tips.

What’s new for 2026 is not more breathless talk of “transformation,” but a clearer picture of what works, what doesn’t, and what still demands adult supervision. The guide now speaks to lawyers who remain leery but are ready to use AI cautiously and competently. It expands beyond first forays to practical, defensible workflows: depositions, motion practice, ESI protocols, voir dire, and making sense of large records without losing the thread. It distinguishes consumer and enterprise tools, explains why governance matters, and emphasizes verification as a professional duty. Crucially, I cover the steps and prompts that get you going. If you’re looking for more hype, this isn’t it. If you want a practical field guide for using AI without surrendering judgment—or credibility—I hope you’ll take a look.

Chambers Guidance: Using AI Large Language Models (LLMs) Wisely and Ethically

19 Thursday Jun 2025

Posted by in ai, General Technology Posts, Law Practice & Procedure

≈ 3 Comments

Tags

, , , , , ,

Tomorrow, I’m delivering a talk to the Texas Second Court of Appeals (Fort Worth), joined by my friend, Lynne Liberato of Houston. We will address LLM use in chambers and in support of appellate practice, where Lynne is a noted authority. I’ll distribute my 2025 primer on Practical Uses for AI and LLMs in Trial Practice, but will also offer something bespoke to the needs of appellate judges and their legal staff–something to-the-point but with cautions crafted to avoid the high profile pitfalls of lawyers who trust but don’t verify.

Courts must develop practical internal standards for the use of LLMs in chambers. These AI applications are too powerful to ignore and too powerful to use without attention given to safe use.

Chambers Guidance: Using AI Large Language Models (LLMs) Wisely and Ethically

Prepared for Second District Court of Appeals (Fort Worth)

Purpose
This document outlines recommended practices for the safe, productive, and ethical use of large language models (LLMs) like ChatGPT-4o in chambers by justices and their legal staff.

I. Core Principles

  1. Human Oversight is Essential
    LLMs may assist with writing, summarization, and idea generation, but should never replace legal reasoning, human editing, or authoritative research.
  2. Confidentiality Must Be Preserved
    Use only secure platforms. Turn off model training/sharing features (“model improvement”) in public platforms or use private/local deployments.
  3. Verification is Non-Negotiable
    Never rely on an LLM for case citations, procedural rules, or holdings without confirming them via Westlaw, Lexis, or court databases.  Every citation is suspect until verified.
  4. Transparency Within Chambers
    Staff should disclose when LLMs were used in a draft or summary, especially if content was heavily generated.  Prompt/output history should be preserved in chambers files.
  5. Judicial Independence and Public Trust
    While internal LLM use may be efficient, it must never undermine public confidence in the independence or impartiality of judicial decision-making. The use of LLMs must not give rise to a perception that core judicial functions have been outsourced to AI.

II. Suitable Uses of LLMs in Chambers

  • Drafting initial outlines of bench memos or summaries of briefs
  • Rewriting judicial prose for clarity, tone, or readability
  • Summarizing long records or extracting procedural chronologies
  • Brainstorming counterarguments or exploring alternative framings
  • Comparing argumentative strength and inconsistencies of and between parties’ briefs

Note: Use of AI output that may materially influence a decision must be identified and reviewed by the judge or supervising attorney.

III. Prohibited or Cautioned Uses

  • Do not insert any LLM-generated citation into a judicial order, opinion, or memo without independent confirmation
  • Do not input sealed or sensitive documents into unsecured platforms
  • Do not use LLMs to weigh legal precedent, assess credibility, or determine binding authority
  • Do not delegate critical judgment or reasoning tasks to the model (e.g., weighing precedent or evaluating credibility)
  • Do not rely on LLMs to generate summaries of legal holdings without human review of the supporting authority

IV. Suggested Prompts for Effective Use

These prompts may be useful when paired with careful human oversight and verification

  • “Summarize this 40-page brief into 5 bullet points, focusing on procedural history.”
  • “Summarize the uploaded transcript respecting the following points….”
  • “Summarize the key holdings and the law in this area”
  • “Rewrite this paragraph for clarity, suitable for a published opinion.”
  • “List potential counterarguments to this position in a Texas appellate context.”
  • “Explain this concept as if to a first-year law student.”

Caution: Prompts seeking legal summaries (e.g., “What is the holding of X?” or “Summarize the law on Y”) are particularly prone to error and must be treated with suspicion. Always verify output against primary legal sources.

V. Public Disclosure and Transparency

Although internal use of LLMs may not require disclosure to parties, courts must be sensitive to the risk that judicial reliance on AI—even as a drafting aid—may be scrutinized. Consider whether and what disclosure may be warranted in rare cases when LLM-generated language substantively shapes a judicial decision.

VI. Final Note

Used wisely, LLMs can save time, increase clarity, and prompt critical thought. Used blindly, they risk error, overreliance, or breach of confidentiality. The justice system demands precision; LLMs can support it—but only under a lawyer’s and judge’s careful eye and hand.

Prepared by Craig Ball and Lynne Liberato, advocating thoughtful AI use in appellate practice.

Of course, the proper arbiters of standards and practices in chambers are the justices themselves; I don’t presume to know better, save to say that any approach that bans LLMs or presupposes AI won’t be used is naive. I hope the modest suggestions above help courts develop sound practical guidance for use of LLMs by judges and staff in ways that promote justice, efficiency and public confidence.

Will AI Summarization Disrupt Discovery?

26 Friday Jan 2024

Posted by in Uncategorized

≈ 6 Comments

Tags

, ,

Reader’s Digest, the century-old magazine with the highest paid circulation, has long published “condensed” books; anthologies of four-to-five popular novels abridged to fit in a single volume.  Condensed Books were once enormously popular, with tens of millions of copies in circulation.  They were also an abomination to serious readers, a literary Tang for those who preferred fresh-squeezed OJ. I’ve never read a condensed book, so I’m in no position to judge their merit save to say that I believe reading anything is a good thing.  I imagine the condensed versions conveyed the guts of the story well enough to sound like you’d read it over drinks with the neighbors before the Ed Sullivan show.

But I am enough of a purist (okay, “snob”) to worry about the impact of summarization.  As an undergraduate English major, I had to wade through some challenging tomes.  I have no empirical evidence for it, but I’m certain those books are a part of me in ways they never would have been had I sought out the Cliffs Notes instead.  I expect most avid readers feel the same.  Summaries necessarily discard content, and what remains is incapable of conveying the same tone, nuance and detail.

So, I worry when the tech industry touts the value of AI summarization of documents, especially as a means of speeding identification and review of evidence in discovery.  I question whether the “Reader’s Digest Condensed Evidence” will convey the same tone, nuance and detail that characterize responsive productions.  Will distillation be made of distillations until genuine intelligence is lost altogether? 

It’s an inchoate apprehension—an old man’s anxiety perhaps—but litigation is about human behavior, human frailty and failings.  I fear too much humanity will disappear in AI-generated summaries with the underlying communications less likely to see the light of day.  The mandate that discovery be “just, speedy and inexpensive” is now read as “just speedy and inexpensive.”  That discarded comma is tragic.

Technology is my lifelong passion.  So, I am not afraid of new tech as much as put off by the embrace of technology to further speed and economy without due consideration of quality.  LegalWeek 2024 will be a carnival of vendors touting AI features and roadmaps.  How many will have metrics to support the quality of their AI-abetted outcomes?  How many have forgotten the comma while chasing the cash? Per Upton Sinclair, ““It is difficult to get a man to understand something, when his salary depends on his not understanding it.”

Unquestionably, we must reduce the cost of discovery to protect the portals of justice.  Justice no one can afford to pursue is no justice at all.  But there are uniquely human characteristics we should continue to esteem in discovery, like curiosity, intuition, suspicion and impression; the “Spidey-sense” we derive from tone, nuance and detail.  Before we use AI to summarize collections then deploy AI to characterize the summaries, can we pause just long enough to see if it’s going to work? Real testing, not just that which supports salaries.