Expert testimony may be needed for e-discovery keyword searches

04 Mar Expert testimony may be needed for e-discovery keyword searches

Editor’s note: This is the first in a regular series of Digital Lex columns by Mark F. Foley, a partner in the Milwaukee based law firm, Foley & Lardner, LLP. The column will discuss legal issues of interest to the technology community, including data privacy and security, data retention and destruction, technology transactions, outsourcing, and electronic discovery.

In February 2008, a U.S. Magistrate Judge issued a decision that may add to the already expensive process of electronic data discovery in litigation. Now expert testimony may be required to determine the adequacy of keyword search terms used to identify and produce electronically stored information, or “ESI.”
Corporate information technology staff and in-house lawyers are often forced to undertake the unenviable task of locating, organizing, searching, and producing all ESI relevant to a dispute. Locating and individually reviewing every e-mail sent or received by a potential witness can be exorbitantly expensive, and sometimes impossible to accomplish in the time permitted. The standard answer to this problem has been to conduct electronic keyword searches against the subject line and body text of the e-mails and other ESI.
The keywords used for such searches are selected in varying ways. Sometimes lawyers for a party requesting information will specify keyword searches. Other times, the producing party will attempt to determine the keywords itself. Often, search terms are negotiated or disputed.
Different priorities
Lawyers, and their clients, may have different goals in mind when developing keyword searches. A requesting party may want to force the opponent to produce a massive amount of data, making the litigation more expensive and increasing the likelihood of settlement. Conversely, a producing party may want to produce massive amounts of data in a difficult form to review, so that the opponent may overlook damaging evidence.
Alternatively, one or both parties may want to locate and produce only the most relevant information, avoiding the need to review large quantities of irrelevant data for privilege, relevancy, and etcetera. Or the parties may not think much about it, with the requesting party asking for search terms developed in an informal brainstorming session and the responding party just looking for whatever the opponent seeks. When the parties disagree about whether a search is sufficiently broad, too narrow, or too expensive to implement, a Magistrate or Judge must decide.
What should the parties expect and how should a judge decide the issue? In U.S. v. Michael John O’Keefe, 2008 WL 449729 (D.D.C.), U.S. Magistrate John Facciola ruled that the adequacy of a keyword search:
“is a complicated question involving the interplay, at least, of the sciences of computer technology, statistics, and linguistics. . . Given this complexity, . . . [t]his topic is clearly beyond the ken of a layman and requires that any such conclusion be based on evidence that, for example, meets the criteria of Rule 702 of the Federal Rules of Evidence.”
Rule 702 is what authorizes the use of expert testimony in Federal cases.
Magistrate Facciola’s use of “for example” should mean that while in all cases testimony from someone would be required to support a contention that search terms were inadequate, the required testimony could be, but did not have to be, expert testimony. The Court banished that sensible approach by explaining:
“Accordingly, if defendants are going to contend that the search terms used by the government were insufficient, . . . their contention must be based on evidence that meets the requirements of Rule 702 of the Federal Rules of Evidence.”
In other words, expert testimony is mandatory.
Although O’Keefe is a criminal case, the Magistrate expressly relied upon cases, standards, and best practice materials relating to discovery of ESI in civil cases. Thus the same rule would apply to common business disputes.
Limits on discovery
On its face, this decision would mean that a motion to compel broader discovery, or to limit discovery that is too broad, based on the quality of the search terms should be dismissed as a matter of law unless it is supported by the testimony of one or more experts in computer science, statistics, and/or linguistics.
That hardly seems appropriate in all circumstances. For example, if the case involved dogs and only “cats” was used as a search term, the obvious inadequacy of the search should be enough to obtain an order to compel, even without expert testimony. Similarly, a search only for data that included “cats AND dogs” is obviously inadequate where any document or ESI containing either “cats OR dogs” could be relevant.
Although the O’Keefe decision goes too far if applied literally and in all cases, it does provide fair warning that expert testimony may be required in some cases. Perhaps all the Magistrate meant is that “If scientific, technical, or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact in issue, a witness qualified as an expert may testify thereto in the form of an opinion or otherwise.” That is the language of Federal Rule of Evidence 702 authorizing the use of opinion testimony by experts.
If the need for expert testimony is based on whether that testimony will in fact assist the trier of fact, Judge Facciola’s ruling is not controversial. If he means as he said, that expert testimony must be provided in every keyword search dispute, this is new territory and another costly addition to the e-discovery process.
You can find more information about the technical aspects of assessing the adequacy of search terms in George L. Paul & Jason R. Baron, Information Inflation: Can the Legal System Adapt? (available in print at 13 RICH. J.L. & TECH. 10 (2007) and from Westlaw online services) and in recent work published by The Sedona Conference.
Drawing the line
Where to draw the line as to what search terms are adequate in a given situation is a matter of technical analysis, decided cases, and experience. If you are involved in, or preparing for litigation involving electronically stored information, you should use litigation counsel with in-depth knowledge about the law and technology involved.
Previous article by Mark Foley
• Mark Foley: Internet law: 12 questions for board oversight of data privacy and security