Reader’s Digest, the century-old magazine with the highest paid circulation, has long published “condensed” books; anthologies of four-to-five popular novels abridged to fit in a single volume. Condensed Books were once enormously popular, with tens of millions of copies in circulation. They were also an abomination to serious readers, a literary Tang for those who preferred fresh-squeezed OJ. I’ve never read a condensed book, so I’m in no position to judge their merit save to say that I believe reading anything is a good thing. I imagine the condensed versions conveyed the guts of the story well enough to sound like you’d read it over drinks with the neighbors before the Ed Sullivan show.
But I am enough of a purist (okay, “snob”) to worry about the impact of summarization. As an undergraduate English major, I had to wade through some challenging tomes. I have no empirical evidence for it, but I’m certain those books are a part of me in ways they never would have been had I sought out the Cliffs Notes instead. I expect most avid readers feel the same. Summaries necessarily discard content, and what remains is incapable of conveying the same tone, nuance and detail.
So, I worry when the tech industry touts the value of AI summarization of documents, especially as a means of speeding identification and review of evidence in discovery. I question whether the “Reader’s Digest Condensed Evidence” will convey the same tone, nuance and detail that characterize responsive productions. Will distillation be made of distillations until genuine intelligence is lost altogether?
It’s an inchoate apprehension—an old man’s anxiety perhaps—but litigation is about human behavior, human frailty and failings. I fear too much humanity will disappear in AI-generated summaries with the underlying communications less likely to see the light of day. The mandate that discovery be “just, speedy and inexpensive” is now read as “just speedy and inexpensive.” That discarded comma is tragic.
Technology is my lifelong passion. So, I am not afraid of new tech as much as put off by the embrace of technology to further speed and economy without due consideration of quality. LegalWeek 2024 will be a carnival of vendors touting AI features and roadmaps. How many will have metrics to support the quality of their AI-abetted outcomes? How many have forgotten the comma while chasing the cash? Per Upton Sinclair, ““It is difficult to get a man to understand something, when his salary depends on his not understanding it.”
Unquestionably, we must reduce the cost of discovery to protect the portals of justice. Justice no one can afford to pursue is no justice at all. But there are uniquely human characteristics we should continue to esteem in discovery, like curiosity, intuition, suspicion and impression; the “Spidey-sense” we derive from tone, nuance and detail. Before we use AI to summarize collections then deploy AI to characterize the summaries, can we pause just long enough to see if it’s going to work? Real testing, not just that which supports salaries.

Pingback: Will AI Summarization Disrupt Discovery? - F1TYM1
Pingback: Week 04 – 2024 – This Week In 4n6
Ralph Artigliere said:
Craig-
Excellent piece. I agree with you about the concern over summaries. As a trial lawyer, I would not rely on anyone else’s paraphrasing of testimony or caselaw. Summarized text was often bent to the version that helped a case, whether it was my side or the opponent. As a judge, I always read the source material to confirm any information that mattered to my decision. With generative AI, results can change depending on the quality of prompting. That alone should give us pause. However, if a lawyer is going to rely on a clerk or paralegal or another lawyer to summarize, AI summaries may be of equal or greater value. I would hope (as you note) that lawyers would have some empirical evidence on whether AI summarization is as good as or better than human summaries before we turn to it in practice. Plus, they need to understand how to use the technology. I may be a dinosaur though. In my last ten years as a trial lawyer, I did not use associates. I preferred to doing my own research, depositions, and trial prep. I did rely on expert assistance to summarize medical records. However, even then, I went to the source to confirm key facts.
LikeLike
craigball said:
Thanks for sharing your thinking and experience, judge. No question summaries have their place and value in litigation, and AI summaries will prove quick and cost-effective. My concern was the narrow one of summaries being used as input to relevance assessment algothritms such that the source material is never reviewed by SMEs and other human assessors. As well, the “Go faster!” assembly line approach to review that has evolved in response to high volumes secondary to inartful culling ensures that (some) reviewers will base relevance determinations solely on a summary, not the source. It’s human nature to seek the path of least resistance–especially when the reviewer won’t be called out for such shortcuts. As Judge Paul Grimm wisely observed, ‘Sometimes the most important part of a decision is in a footnote, and that will be the first to go in an AI summary.’
LikeLike
artiglierer said:
Thank you for the quick and instructive response. I am fully on board with your reasoning, and I hope it gains the traction it deserves.
LikeLike
Pingback: Will AI Summarization Disrupt Discovery? Here’s What Craig Ball Thinks