If you talk frankly with those in the predictive coding business, it won’t be long until they lament, “Our beans are so magical, we just can’t fathom why more customers aren’t planting them.” They insist that lawyers (or corporations or judges) don’t “get it.” That is, those who hold the purse strings or call the shots don’t appreciate how much money can be saved, how much better the result can be or how much tedious review can be delegated to tireless technology tools.
It’s a common sentiment expressed by all in the business of selling technology assisted review; yet despite their shared frustration, I see no indication that they have laid down their arms and formed an industry association geared to jointly advancing shared goals and resolving common problems The result has been widespread confusion about the technology and more heat than light when it comes to litigants having the confidence to not only use the emerging technologies but also to deploy them in ways that don’t trade one broken system for another.
It would be much easier to educate the bench and bar from a place of consensus; and cheaper, too. As the small cadre of providers claw for their piece of a nascent market, they sow seeds of doubt. The statistical precepts are identical. The underlying technologies are substantially similar (although the particulars of the algorithms differ). The tools have more in common than sellers care to admit. Certainly, consumers think them much the same.
When the dairy industry wanted to float all boats, they asked, simply, “Got Milk?” They didn’t ask “Got Milk from from Holstein-Friesian cattle milked in herringbone parlors and sterilized by plate heat exchangers?“
How is it a “Got Milk?” message that didn’t promote the interests of one supplier over another acquired a 90% awareness among U.S. consumers? Could it be that members of milk boards put their common needs above their individual market share?
“But,” the sellers reply, “milk is a fungible good, and e-discovery services aren’t!”
Hmmm, Really? Does it seem to you, dear reader, that consumers of e-discovery services seek highly-differentiated offerings, not commoditized services? Are no suppliers being shut out by low bidders touting meat cleaver culling at rock-bottom per gigabyte pricing when technology-assisted approaches would be so much better?
It’s time to wake up and smell the TAR. Predictive coding vendors (and others) need to band together, establish practical guidelines and reasonable standards, jointly promote education and research and see where a little shared effort can build a market big enough for all–not to fix prices or divvy up spoils, but to direct energy and creativity to secure a whole greater than the sum of its parts and to prevent quality from becoming the red-headed stepchild of EDD.
So, Bob, Craig, Arnaud, Quin, Herb, Steve, Ian, Jim, Amir, Warwick, Nicholas, and perhaps even you, too, Gordon, Maura, Bill and Karl. Don’t wait for Sedona, EDRM, EDI or (heaven help us) one of the EDD lobbying groups to call the tune! Form your own industry organization now. Check your egos at the door, keep it simple, don’t fight over who will run it, have an organizational meeting, identify common interests, hire a part-time admin and write some modest checks. The payback to you will be great, and the benefit to the justice system greater still.
And, no, there’s nothing in it for me. I just think it’s a good idea. Apologies to those whose names I omitted in my desire to avoid sounding like the start of a Mickey Mouse Club episode.
Ball in your Court 
Deborah Ratterman-Warnecke said:
Well said and couldn’t agree more!
LikeLike
Pingback: Digital Forensics, Inc. Got TAR? | Digital Forensics, Inc.
James Q said:
Regarding educating the bench and bar – perhaps these two entities demand two education strategies.
For the bench, it should be enough that we prove that the not-produced set has no responsive documents in it. Unless cost-shifting is in play, it should not matter how we get there. If a party wants to review every document by a human reading every word, they can. If a party wants to hire a team of robots they can. The end result must be the same, that documents supporting claims and defenses or which are responsive to requests are produced. Success in this regard is demonstrated by sampling the withheld set. Sample it until the cows come home, if the techniques to get there worked, the bench will be won.
But the bar needs to be convinced how to get there cheapest. That is a simplification of course, but that is where the products differentiate. One is cheaper, one is more accurate, one is faster, one is less risky.
The problem with “band(ing) together” to “build a market” is that the product (TAR) is young enough that the market is (by definition) yet to be made. This puts any individual provider in a place to make the market. If I can make the market, and I have not given up on that, why would I band together with someone else? Banding together will come later, once a leader has emerged.
LikeLike
craigball said:
James Q:
Would that it were so simple, but the standard is not–and likely never will be–that one must “prove that the not-produced set has no responsive documents in it.” Anyone who studies the performance of the tools and workflows appreciates that, in a collection of any significant size, the null documents will always hold a complement of responsive documents. For example, when keywords alone are used to cull the collection in a single iteration, one can expect to leave 2-4 responsive documents in the null set for every one produced. Even with so-called “best practices” in place, there are no perfect parsings of responsive documents.
More likely, courts will demand proof that the tools and processes employed to identify, collect, segregate and produce responsive documents were reasonably calculated to do the job. By that standard, it’s wise to expect courts to look at the methods, sampling and quality assurance/control mechanisms employed. An industry group can help with that effort and collectively benefit from a better-informed bench taught to ask the right questions and expect certain competent and consistent answers.
LikeLike
James Q said:
Fair enough, I should not have said “no responsive documents” but something along the lines of “only a reasonable number of responsive documents” or something that included the word “proportional.” But again, as you say, the “sampling and quality assurance/control mechanisms” applies to the null set, I have not seen courts concerning themselves with how a party got (ex post) the null set to meet acceptable criteria. It’s the client that cares, because of that whole money thing.
LikeLike
ediscoverystories said:
and yet one more thing that could benefit from a little more cooperation and a little less profit and selfishness – well said, Craig!
LikeLike
Pingback: Analysis of the Official Report on the 2011 TREC Legal Track – Part One « e-Discovery Team ®
Pingback: The Many Faces of Mike McBride » Blog Archive » This Week’s Links (weekly)
Tony Patrick (@Patrick4n6) said:
There’s no problem with TAR that a successful Daubert hearing wouldn’t fix.
LikeLike
Michael Fluhr said:
Great post.
In addition to the points above, an industry group might also market TAR as a tool for requesting (rather than producing) parties. I’ve begun to suspect that the TAR industry has targeted the wrong consumer group. As far as I can tell, the industry has marketed TAR to potential producing parties as a cheaper way to find and produce a greater percentage of relevant documents. To a techie, this sounds great. To an attorney, however, not so much. The attorney asks, “Why would I or my client want to use a technology that finds more relevant documents for production?” The answer is not clear. For this reason, I increasingly find it unsurprising that producing parties haven’t all rushed to the TAR market.
Perhaps the better marketing target consists of litigants (and firms) in a position to demand TAR use from opposing parties. Just as litigants have demanded full-text searches from opposing parties, so too might they demand TAR searches (with transparent protocols). A flood of TAR demand might finally come not from the producers, but the requesters.
Here, an industry group makes sense. Individual TAR vendors appear unlikely to market to litigants in a position to demand TAR from opposing parties; these litigants lack the position to select the paticular vendor’s TAR product over other products (a position really the purview of the producing party). But a coordinated industrywide push might give each TAR vendor a better return on its marketing investment. It might also synergistically allow a greater marketing reach than the same investment made in an uncoordinated fashion by individual vendors.
Ultimately, I’m eager to see how the TAR industry will solve these marketing problems. I think most of us agree that TAR is cheaper and more efficient that its alternatives and that this should sell itself, to litigants and courts. The fact that it hasn’t yet, at least in mass numbers, suggests something is missing in the marketing.
LikeLike
Sandy Serkes said:
Craig, there have been those of us who have tried to form such cross-company, industry-oriented alliances, only to be shut-down due to size, budget realities and customer conflict. In an industry where a technology or capability provider must “pay to play” in order to have their voice heard, is it any wonder there is a distinct advantage to keeping smaller, less-funded voices quiet? You, Craig, are in a position to beat this system. Organize a panel (or better yet a real-time bake-off) that takes all comers who are willing to put forth technology, time, effort, and ideas, not just marketing dollars and rhetoric. I bet you would hear from some very different quadrants – not the Bobs, Warwicks & Nicholases, but the Sandys, the Johns (Martin), the Gerrys and who knows who else is out there? Your own blog post answers your question.
LikeLike
Pingback: Technology-Assisted Review: From Expert Mentions to Mainstream Coverage | @ComplexD
Pingback: Review Less, LLC - The “E’s” of Predictive Coding- Part One