Computer Assisted Review

Computer Assisted Review (CAR).

Intro. Review of documents with the assistance of computers and specially designed legal search and review software is a well established best practice. If you have more than a thousand documents it simply makes no sense to print them all out and review and mark them by hand without computer assistance. But the term Computer Assisted Review, and the alternative phrase that has the same meaning, Technology Assisted Review, has come to mean more than simply reviewing and coding documents on a computer. It now indicates a process where attorneys use  computer software as indispensable tools to search and find relevant evidence in a big data setting. This is the needle in the haystack problem that is a core issue of e-discovery.

Due to the high volume of ESI that may be relevant to a dispute, even after the Bulk Culling processes described here, it is impractical for individual review of all custodian documents that might be relevant. CAR now means to use software for the review and coding documents, and also for the further search and culling down of a dataset to the most likely relevant documents for final attorney review.

Individual attorney review of all documents that survive Bulk Culling is not a best practice. It is not only exorbitantly expensive, even with the use of low paid contract review attorneys, it is also less effective. This is now well established, not only by extensive attorney experience, but my numerous scientific studies. Grossman, Cormack, Technology-assisted review in e-discovery can be more effective and more efficient than exhaustive manual review, Richmond Journal of Law and Technology, 17(3):11:1–48 (2011); Roitblat, Kershaw, and Oot, Document categorization in legal electronic discovery: computer classification vs. manual review, Journal of the American Society for Information Science and Technology, 61(1):70–80 (2010); Webber, W. Re-examining the Effectiveness of Manual Review (2011); Ellen M. Voorhees, Variations in relevance judgments and the measurement of retrieval effectiveness, 36:5 Information Processing & Management 697, 701 (2000).

See NYSBA Best Practices In E-Discovery In New York State and Federal Courts (2011) (Guideline 8 contains a good high-level description of the review process; and 10 addresses scope of review and reasonability of search techniques utilized). Also see: The Sedona Conference® Best Practices Commentary on Search & Retrieval Methods, August 2007 (look for updated final version soon).

The CAR best practices are broken down into these sub-pages:

For more background on CARs, specifically those with predictive coding type search engines, see the e-Discovery Team CAR page.

It is Ralph Losey’s radically conservative view that the best practice in CAR is for the raw unreviewed data to be hosted by qualified vendors, and not kept on a law firm’s own computer system. Instead, he recommends that lawyers and law firms only have actual possession of the evidence obtained after review of the bulk collections. For details on his recommended best practice of outsourcing possession to qualified vendors, instead of law firm Litigation Support Departments, and the full rationale therefore, see Best Practices in e-Discovery for Handling Large Stores of Unreviewed Client Data.

If you have any suggestions and care to contribute to this project, or any questions (nothing case specific please), please leave a comment below.