Review activities: the green squares in EDBP represent search and review related legal services – Culling, C.A.R. and Protections. These legal services typically constitute the majority of the expense in e-discovery litigation. Click on the links of the bullet-points for the sub-pages that contain the descriptions of these review related best practices.

  • Protections of Privacy and Confidentiality (to be written)

Planning for a large review project is critical to its success, especially when predictive coding will be used. Here is a detailed 12-step Outline of a Form Plan of a Predictive Coding ProjectAlso see the work of the Coalition of Technology Resources for Lawyers (CTRL), especially their Model Stipulation and Order for use of predictive coding.

Also see NYSBA Best Practices In E-Discovery In New York State and Federal Courts (2011) (Guideline 11 on search and review to protect privileged ESI and clawback type agreements).

Also See Judge Paul Grimm’s standing Discovery Order, for cases before him in the District Court in Maryland. In mid 2015 he added this language in paragraph 6.b.i. on predictive coding:

Parties requesting ESI discovery and parties responding to such requests are expected to cooperate in the development of search methodology and criteria to achieve proportionality in ESI discovery, including appropriate use of computer-assisted search methodology, such as Technology Assisted Review, which employs advanced analytical software applications that can screen for relevant, privileged, or protected information in ways that are more accurate than manual review and involve far less expense.

Judge Grimm’s Discovery Order at paragraph 9 automatically imposes a Rule 502(d), Federal Evidence Code, mandate that an unintentional disclosure of privileged materials shall “not  constitute a waiver in this proceeding, or in any other federal or state proceeding.” This wording in the order makes the gotcha provisions of 502(b)(2) inapplicable. Here is the full exact wording:

In accordance with Fed. R. Evid. 502(d), except when a party intentionally waives attorney–client privilege or work product protection by disclosing such information to an adverse party as provided in Fed. R. Evid. 502(a), the disclosure of attorney–client privileged or work product protected information pursuant to a non-waiver agreement entered into under Fed. R. Evid. 502(e) does not constitute a waiver in this proceeding, or in any other federal or state proceeding. Further, the provisions of Fed. R. Evid. 502(b)(2) are inapplicable to the production of ESI pursuant to an agreement entered into between the parties under Fed. R. Evid. 502(e). . . .

We agree with Judge Scheindlin, who, with Judge Grimm by her side at a CLE several years ago after 502 was enacted, wondered aloud why all judges did not simply enter a 502(d) order in all of their cases, instead of waiting for lawyers to do it, and perhaps screw up in the process. Now Judge Grimm has done just that. He enters the 502(d) broad scope claw back protection order sua sponte, and does so, as far as we can see, in all cases. A handful of other judges around the country are starting to do the same. In our opinion this is a best practice for all federal judges to follow. State judges should do it too, but without reference to the federal evidence code.

Count yourself lucky if you are one of the few lawyers in the country with an enlightened judge like that who is attempting to rehabilitate and use the often misunderstood provisions of clawback Rule 502. If you are in the vast majority, and your judge does not have a standing 502(d) order, then by all means raise the issue. Move for blanket protection under Rule 502(d) and avoid the reasonable efforts gotcha provisions of 502(b)(2) (“the holder of the privilege or protection took reasonable steps to prevent disclosure”). See eg. Innovate Conference Report: Judge Facciola and Judge Nolan to Speak on Clawbacks, Cooperation and CompetenceAnother Clawback Enforcement Order Shows the Importance of the Selection of Quality Vendorsand, Clawbacks: Trick or Treat?.

More guidelines on last section on Privacy and Confidentiality will be coming. Please propose best practice description of these and other Review areas in the comments below, or by email to Ralph Losey.

General. The Search and Review activities – 6, 7, 8 – are intimately connected with and flow out of Cooperation, #5. See diagram below. The green Review steps are typically oriented to Productions of documents to requesting parties, but not always. Some search and review lead only to internal Productions, such as investigations. Further, the Protections step in Review – 8 – is actually designed for non-production of ESI that does not have to be produced, such as non-responsive or privileged documents. Again, this ties back to the Cooperation step where scope of relevancy or responsivity is determined.

Although not included in the diagram slice above, it is worth noting that Step 3, Custodian/Witness Interviews, is also implicated in the Keyword Search component of Hybrid Multimodal C.A.R.. William A. Gross Construction Associates, Inc. v. American Manufacturers Mutual Insurance Co., 256 F.R.D. 134, 136 (S.D.N.Y. 2009).

The e-DiscoveryTeam blog has over sixty articles on best practices in search. The chart below summarizes the Team’s latest insights on their method, which they call Predictive Coding 4.0. For a complete elucidation of the methods see the 97 page essay in PDF formPredictive Coding 4.0 – Nine Key Points of Legal Document Review and an Updated Statement of Our Workflow. The same essay in html form can be found that includes ten videos to supplement the written words.


For an earlier summary of the Team’s search insights see the below graphic using the Olympic rings.

The vertical chain of five rings represents the five basic methods of legal search. The five horizontal rings represent five basic insights into search that underlie best practices in review. The interlocking ring of Hybrid Modal, represents the most effective type of search today, that includes all five of the vertical rings. For more background on these ten concepts see: Day Ten of a Predictive Coding Narrative: A post hoc test of my hypothesis of insignificant false negatives.


If you have any suggestions on best practices for Review, and care to contribute to this project, or any questions (nothing case specific please), please leave a comment below or email Ralph Losey directly.


One thought on “Review

  1. Submitted by email from a practicing attorney in Manhattan:

    Cooperation/Search and Review
    1. I am seeing increasing need for meta-data and think maybe it may deserve its own line-item here (But I agree that it would/could be part of any number of already existing categories)
    2. “Computer Assisted” … I suggest “Technology Assisted” as an alternative in light of the rapid pace of tech advances …”computer” may be obsolete soon
    3. I suggest adding a “Confidential Data” or “Sensitive Data” line-item as I’m not sure that sensitive proprietary client info would be covered under “Privacy Protections”

Please leave a comment and help improve EDBP