• Home
  • About
  • CRAIGBALL.COM
  • Disclaimer
  • Log In

Ball in your Court

~ Musings on e-discovery & forensics.

Ball in your Court

Tag Archives: artificial-intelligence

A Master Table of Truth

04 Tuesday Nov 2025

Posted by craigball in ai, Computer Forensics, E-Discovery, General Technology Posts, Law Practice & Procedure, Uncategorized

≈ 5 Comments

Tags

ai, artificial-intelligence, chatgpt, eDiscovery, generative-ai, law, technology

Lawyers using AI keep turning up in the news for all the wrong reasons—usually because they filed a brief brimming with cases that don’t exist. The machines didn’t mean to lie. They just did what they’re built to do: write convincingly, not truthfully.

When you ask a large language model (LLM) for cases, it doesn’t search a trustworthy database. It invents one. The result looks fine until a human judge, an opponent or an intern with Westlaw access, checks. That’s when fantasy law meets federal fact.

We call these fictions “hallucinations,” which is a polite way of saying “making shit up;” and though lawyers are duty-bound to catch them before they reach the docket, some don’t. The combination of an approaching deadline and a confident-sounding computer is a dangerous mix.

Perhaps a Useful Guardrail

It struck me recently that the legal profession could borrow a page from the digital forensics world, where we maintain something called the NIST National Software Reference Library (NIST NSRL). The NSRL is a public database of hash values for known software files. When a forensic examiner analyzes a drive, the NSRL helps them skip over familiar system files—Windows dlls and friends—so they can focus on what’s unique or suspicious.

So here’s a thought: what if we had a master table of genuine case citations—a kind of NSRL for case citations?

Picture a big, continually updated, publicly accessible table listing every bona fide reported decision: the case name, reporter, volume, page, court, and year. When your LLM produces Smith v. Jones, 123 F.3d 456 (9th Cir. 2005), your drafting software checks that citation against the table.

If it’s there, fine—it’s probably references a genuine reported case.
If it’s not, flag it for immediate scrutiny.

Think of it as a checksum for truth. A simple way to catch the most common and indefensible kind of AI mischief before it becomes Exhibit A at a disciplinary hearing.

The Obstacles (and There Are Some)

Of course, every neat idea turns messy the moment you try to build it.

Coverage is the first challenge. There are millions of decisions, with new ones arriving daily. Some are published, some are “unpublished” but still precedential, and some live only in online databases. Even if we limited the scope to federal and state appellate courts, keeping the table comprehensive and current would be an unending job; but not an insurmountable obstacle.

Then there’s variation. Lawyers can’t agree on how to cite the same case twice. The same opinion might appear in multiple reporters, each with its own abbreviation. A master table would have to normalize all of that—an ambitious act of citation herding.

And parsing is no small matter. AI tools are notoriously careless about punctuation. A missing comma or swapped parenthesis can turn a real case into a false negative. Conversely, a hallucinated citation that happens to fit a valid pattern could fool the filter, which is why it’s not the sole filter.

Lastly, governance. Who would maintain the thing? Westlaw and Lexis maintain comprehensive citation data, but guard it like Fort Knox. Open projects such as the Caselaw Access Project and the Free Law Project’s CourtListener come close, but they’re not quite designed for this kind of validation task. To make it work, we’d need institutional commitment—perhaps from NIST, the Library of Congress, or a consortium of law libraries—to set standards and keep it alive.

Why Bother?

Because LLMs aren’t going away. Lawyers will keep using them, openly or in secret. The question isn’t whether we’ll use them—it’s how safely and responsibly we can do so.

A public master table of citations could serve as a quiet safeguard in every AI-assisted drafting environment. The AI could automatically check every citation against that canonical list. It wouldn’t guarantee correctness, but it would dramatically reduce the risk of citing fiction. Not coincidentally, it would have prevented most of the public excoriation of careless counsel we’ve seen.

Even a limited version—a federal table, or one covering each state’s highest court—would be progress. Universities, courts, and vendors could all contribute. Every small improvement to verifiability helps keep the profession credible in an era of AI slop, sloppiness and deep fakes.

No Magic Bullet, but a Sensible Shield

Let’s be clear: a master table won’t prevent all hallucinations. A model could still misstate what a case holds, or cite a genuine decision for the wrong proposition. But it would at least help keep the completely fabricated ones from slipping through unchecked.

In forensics, we accept imperfect tools because they narrow uncertainty. This could do the same for AI-drafted legal writing—a simple checksum for reality in a profession that can’t afford to lose touch with it.

If we can build databases to flag counterfeit currency and pirated software, surely we can build one to spot counterfeit law?

Until that day, let’s agree on one ironclad proposition: if you didn’t verify it, don’t file it.

Share this:

  • Click to email a link to a friend (Opens in new window) Email
  • Click to print (Opens in new window) Print
  • Click to share on X (Opens in new window) X
  • Click to share on Facebook (Opens in new window) Facebook
  • Click to share on LinkedIn (Opens in new window) LinkedIn
Like Loading...

Native or Not? Rethinking Public E-Mail Corpora for E-Discovery (Redux, 2013→2025)

16 Saturday Aug 2025

Posted by craigball in ai, Computer Forensics, E-Discovery, Uncategorized

≈ 2 Comments

Tags

ai, artificial-intelligence, chatgpt, eDiscovery, EDRM, generative-ai, Linked attachments, Purview, technology

Yesterday, I found myself in a spirited exchange with a colleague about whether the e-discovery community has suitable replacements for the Enron e-mail corpora1—now more than two decades old—as a “sandbox” for testing tools and training students. I argued that the quality of the data matters: native or near-native e-mail collections remain essential to test processing and review workflows in ways that mirror real-world litigation.

The back-and-forth reminded me that, unlike forensic examiners or service providers, ediscovery lawyers may not know or care much about the nature of electronically-stored information until it finds its way to a review tool. I get that. If your interest in email is in testing AI coding tools, you’re laser-focused on text and maybe a handful of metadata; but if your focus is on the integrity and authenticity of evidence, or in perfecting processing tools, the originating native or near-native form of the corpus matters more.

What follows is a re-publication of a post from July 2013. I’m bringing it back because the debate over forms of email hasn’t gone away; the issue is as persistent and important as ever. A central takeaway bears repeating: the litmus test is whether a corpus hews to a fulsome RFC-5322 compliant format. If headers, MIME boundaries, and transport artifacts are stripped or incompletely synthesized, what remains ceases to be a faithful native or near-native format. That distinction matters, because even experienced e-discovery practitioners—those fixated on review at the far-right side of the EDRM—may not fully appreciate what an RFC-5322 email is, or how much fidelity is lost when working with post-processed sets.

Continue reading →

Share this:

  • Click to email a link to a friend (Opens in new window) Email
  • Click to print (Opens in new window) Print
  • Click to share on X (Opens in new window) X
  • Click to share on Facebook (Opens in new window) Facebook
  • Click to share on LinkedIn (Opens in new window) LinkedIn
Like Loading...

Chambers Guidance: Using AI Large Language Models (LLMs) Wisely and Ethically

19 Thursday Jun 2025

Posted by craigball in ai, General Technology Posts, Law Practice & Procedure

≈ 3 Comments

Tags

ai, artificial-intelligence, chatgpt, generative-ai, law, LLM, technology

Tomorrow, I’m delivering a talk to the Texas Second Court of Appeals (Fort Worth), joined by my friend, Lynne Liberato of Houston. We will address LLM use in chambers and in support of appellate practice, where Lynne is a noted authority. I’ll distribute my 2025 primer on Practical Uses for AI and LLMs in Trial Practice, but will also offer something bespoke to the needs of appellate judges and their legal staff–something to-the-point but with cautions crafted to avoid the high profile pitfalls of lawyers who trust but don’t verify.

Courts must develop practical internal standards for the use of LLMs in chambers. These AI applications are too powerful to ignore and too powerful to use without attention given to safe use.

Chambers Guidance: Using AI Large Language Models (LLMs) Wisely and Ethically

Prepared for Second District Court of Appeals (Fort Worth)


Purpose
This document outlines recommended practices for the safe, productive, and ethical use of large language models (LLMs) like ChatGPT-4o in chambers by justices and their legal staff.


I. Core Principles

  1. Human Oversight is Essential
    LLMs may assist with writing, summarization, and idea generation, but should never replace legal reasoning, human editing, or authoritative research.
  2. Confidentiality Must Be Preserved
    Use only secure platforms. Turn off model training/sharing features (“model improvement”) in public platforms or use private/local deployments.
  3. Verification is Non-Negotiable
    Never rely on an LLM for case citations, procedural rules, or holdings without confirming them via Westlaw, Lexis, or court databases.  Every citation is suspect until verified.
  4. Transparency Within Chambers
    Staff should disclose when LLMs were used in a draft or summary, especially if content was heavily generated.  Prompt/output history should be preserved in chambers files.
  5. Judicial Independence and Public Trust
    While internal LLM use may be efficient, it must never undermine public confidence in the independence or impartiality of judicial decision-making. The use of LLMs must not give rise to a perception that core judicial functions have been outsourced to AI.

II. Suitable Uses of LLMs in Chambers

  • Drafting initial outlines of bench memos or summaries of briefs
  • Rewriting judicial prose for clarity, tone, or readability
  • Summarizing long records or extracting procedural chronologies
  • Brainstorming counterarguments or exploring alternative framings
  • Comparing argumentative strength and inconsistencies of and between parties’ briefs

Note: Use of AI output that may materially influence a decision must be identified and reviewed by the judge or supervising attorney.


III. Prohibited or Cautioned Uses

  • Do not insert any LLM-generated citation into a judicial order, opinion, or memo without independent confirmation
  • Do not input sealed or sensitive documents into unsecured platforms
  • Do not use LLMs to weigh legal precedent, assess credibility, or determine binding authority
  • Do not delegate critical judgment or reasoning tasks to the model (e.g., weighing precedent or evaluating credibility)
  • Do not rely on LLMs to generate summaries of legal holdings without human review of the supporting authority

IV. Suggested Prompts for Effective Use

These prompts may be useful when paired with careful human oversight and verification

  • “Summarize this 40-page brief into 5 bullet points, focusing on procedural history.”
  • “Summarize the uploaded transcript respecting the following points….”
  • “Summarize the key holdings and the law in this area”
  • “Rewrite this paragraph for clarity, suitable for a published opinion.”
  • “List potential counterarguments to this position in a Texas appellate context.”
  • “Explain this concept as if to a first-year law student.”

Caution: Prompts seeking legal summaries (e.g., “What is the holding of X?” or “Summarize the law on Y”) are particularly prone to error and must be treated with suspicion. Always verify output against primary legal sources.


V. Public Disclosure and Transparency

Although internal use of LLMs may not require disclosure to parties, courts must be sensitive to the risk that judicial reliance on AI—even as a drafting aid—may be scrutinized. Consider whether and what disclosure may be warranted in rare cases when LLM-generated language substantively shapes a judicial decision.

VI. Final Note

Used wisely, LLMs can save time, increase clarity, and prompt critical thought. Used blindly, they risk error, overreliance, or breach of confidentiality. The justice system demands precision; LLMs can support it—but only under a lawyer’s and judge’s careful eye and hand.


Prepared by Craig Ball and Lynne Liberato, advocating thoughtful AI use in appellate practice.

Of course, the proper arbiters of standards and practices in chambers are the justices themselves; I don’t presume to know better, save to say that any approach that bans LLMs or presupposes AI won’t be used is naive. I hope the modest suggestions above help courts develop sound practical guidance for use of LLMs by judges and staff in ways that promote justice, efficiency and public confidence.

Share this:

  • Click to email a link to a friend (Opens in new window) Email
  • Click to print (Opens in new window) Print
  • Click to share on X (Opens in new window) X
  • Click to share on Facebook (Opens in new window) Facebook
  • Click to share on LinkedIn (Opens in new window) LinkedIn
Like Loading...

Leery Lawyer’s Guide to AI

09 Thursday Jan 2025

Posted by craigball in E-Discovery, General Technology Posts

≈ 10 Comments

Tags

ai, artificial-intelligence, chatgpt, eDiscovery, generative-ai, openai, technology

Next month, I’m privileged to be presenting on two topics with United States District Judge Xavier Rodriguez, a dear friend who sits in the Western District of Texas (San Antonio). One of those topics is “Practical Applications for AI.” The longstanding custom for continuing legal education in Texas is that a presenter must offer “high quality written materials” to go with a talk. I’m indebted to this obligation because writing is hard work and without the need to supply original scholarship, I’d probably have produced a fraction of what I’ve published over forty years. A new topic meant a new paper, especially as I was the proponent of the topic in the planning stage–an ask borne of frustration. After two years of AI pushing everything else aside, I was frustrated by the dearth of practical guidance available to trial lawyers–particularly seasoned elders–who want to use AI but fear looking foolish…or worse. So, I took a shot at a practical primer for litigators and am reasonably pleased with the result. Download it here. For some it will be too advanced and for others too basic; but I’m hopeful it hits the sweet spot for many non-technical trial lawyers who don’t want to be left behind.

Despite high-profile instances of lawyers getting into trouble by failing to use LLMs responsibly, there’s a compelling case for using AI in your trial practice now, even if only as a timesaver in document generation and summarization—tasks where AI’s abilities are uncanny and undeniable. But HOW to get started?

The Litigation Section of the State Bar of Texas devoted the Winter 2024 issue of The Advocate magazine to Artificial Intelligence.  Every article was well-written and well-informed—several penned by close friends—but no article, not one, was practical in the sense of helping lawyers use AI in their work. That struck me as an unmet need.

As I looked around, I found no articles geared to guiding trial lawyers who want to use LLMs safely and strategically. I wanted to call the article “The Leery Lawyer’s Guide to AI,” but I knew it would be insufficiently comprehensive. Instead, I’ve sought to help readers get started by highlighting important considerations and illustrating a few applications that they can try now with minimal skill, anxiety or expense. LLMs won’t replace professional judgment, but they can frame issues, suggest language, and break down complex doctrines into plain English explanations. In truth, they can do just about anything that a mastery of facts and language can achieve.

But Know This…

LLMs are unlike any tech tool you’ve used before. Most of the digital technology in our lives is characterized by consistency: you put the same things in, and other things come out in a rigid and replicable fashion. Not so with LLMs. Ask ChatGPT the same question multiple times, and you’ll get a somewhat different answer each time. That takes getting used to. 

Additionally, there’s no single “right” way to interrogate ChatGPT to be assured of an optimal result. That is, there is no strict programming language or set of keywords calculated to achieve a goal. There are a myriad number of ways to successfully elicit information from ChatGPT, and in stark contrast to the inflexible and unforgiving tech tools of the past, the easiest way to get the results you want is to interact with ChatGPT in a natural, conversational fashion.

Continue reading →

Share this:

  • Click to email a link to a friend (Opens in new window) Email
  • Click to print (Opens in new window) Print
  • Click to share on X (Opens in new window) X
  • Click to share on Facebook (Opens in new window) Facebook
  • Click to share on LinkedIn (Opens in new window) LinkedIn
Like Loading...
Follow Ball in your Court on WordPress.com

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 2,223 other subscribers

Recent Posts

  • A Master Table of Truth November 4, 2025
  • Kaylee Walstad, 1962-2025 August 19, 2025
  • Native or Not? Rethinking Public E-Mail Corpora for E-Discovery (Redux, 2013→2025) August 16, 2025
  • Still on Dial-Up: Why It’s Time to Retire the Enron Email Corpus August 15, 2025
  • Chambers Guidance: Using AI Large Language Models (LLMs) Wisely and Ethically June 19, 2025

Archives

RSS Feed RSS - Posts

CRAIGBALL.COM

Helping lawyers master technology

Categories

EDD Blogroll

  • eDiscovery Today (Doug Austin)
  • Complex Discovery (Rob Robinson)
  • The Relativity Blog
  • E-Discovery Law Alert (Gibbons)
  • GLTC (Tom O'Connor)
  • Sedona Conference
  • Basics of E-Discovery (Exterro)
  • Illuminating eDiscovery (Lighthouse)
  • CS DISCO Blog
  • Minerva 26 (Kelly Twigger)
  • Corporate E-Discovery Blog (Zapproved )
  • E-D Team (Ralph Losey)
  • eDiscovery Journal (Greg Buckles)

Admin

  • Create account
  • Log in
  • Entries feed
  • Comments feed
  • WordPress.com

Enter your email address to follow Ball in Your Court and receive notifications of new posts by email.

Website Powered by WordPress.com.

  • Subscribe Subscribed
    • Ball in your Court
    • Join 2,075 other subscribers
    • Already have a WordPress.com account? Log in now.
    • Ball in your Court
    • Subscribe Subscribed
    • Sign up
    • Log in
    • Report this content
    • View site in Reader
    • Manage subscriptions
    • Collapse this bar
 

Loading Comments...
 

    %d