edrm-IDIs anyone else troubled that the most oft-cited research into e-discovery–the Blair & Marron study of keyword search–dates from 1985? Recent “studies” are often seat-of-the-pants opinion polls of the sort that ask in house counsel to guess how well prepared their companies are to deal with e-discovery or what they think discovery costs. These are interesting; but, they’re no more reliable than polls asking people to rate themselves as “fair minded” or “intelligent.” Polls measure people’s expectations about what facts might be, not facts. The long-held consensus that the sun circled a flat Earth didn’t make it so.

We need objective metrics in e-discovery, and one thing I’d like to see measured is the origin of the information obtained in discovery that’s actually used to prosecute or defend cases. My experience is that cases are won or lost using a handful of items versus the number exchanged in discovery. Do the exhibits used in motions, depositions and trials derive from e-discovery or do they emerge by other means?

I imagine such a study might look at, say, a dozen recent cases that went to verdict. These would be distributed across different types of lawsuits, perhaps four IP disputes, four personal injury matters and four business cases. Each item used as an exhibit to a motion, at an oral deposition or in trial would be traced back through discovery to determine how it was identified. Was it ESI? Was it found by electronic search using keywords? If so, which side proffered the keyword(s) that found it? Was it a scan of legacy paper records, elicited from a database, derived from e-mail, found by forensics or subpoenaed from third parties? Did its disclosure follow a motion to compel? Was it something the requesting party had before suit?

There is no consensus about the value of e-discovery versus its cost. We may learn that discovery poorly correlates to probative evidence, that it’s more valuable in certain types of cases than in others or that without e-discovery, litigants would be deprived of crucial evidence. Perhaps it will develop that it’s misleading to look at cases tried when the greatest value of discovery could flow from the cases it helps to settle. I have an opinion; but I don’t know. I’d like to know. Wouldn’t you?