Unreported / Non-Citable
Background
Plaintiffs brought this securities class action in the Northern District of California against Alphabet Inc., Sundar Pichai, and Google LLC, alleging that defendants violated the Securities Exchange Act by making misleading statements concerning Google's digital advertising technology business. During discovery, the parties reached an impasse over the appropriate protocol for reviewing and producing custodial documents collected from two groups: twenty-seven custodians whose files had previously been produced in related ad tech antitrust litigation underlying plaintiffs' suit, and twenty newly added custodians. Earlier statistical sampling using plaintiffs' initial search terms generated low hit rates of 0.25% across the twenty-seven prior custodians and 0.06% across the twenty new custodians.
Plaintiffs subsequently demanded that defendants conduct statistical sampling on an individual custodian basis using revised search terms. During meet-and-confer discussions, defendants initially agreed to draw random samples at a 95% confidence level with a 5% margin of error across each of the forty-seven custodians, amounting to approximately 400 documents per custodian and nearly 20,000 documents in total. However, in the joint discovery letter submitted to the court, defendants changed position and argued that no protocol was necessary because further random sampling was unwarranted.
In the alternative, the parties presented competing proposals regarding the operational rules of any sampling protocol. Plaintiffs requested a 10% responsiveness threshold triggering document review and production, a null set sample of non-responsive documents to test categorization accuracy, a narrower 2.5% margin of error, and disclosure of recall rates. Defendants countered that any automatic review threshold should be set at 15%, with custodians yielding between 10% and 15% responsiveness reserved for further meet-and-confer discussions. Defendants also opposed producing a null set, arguing that null set validation is limited to technology-assisted review rather than keyword searches.
The Court’s Holding
United States Magistrate Judge Kandis A. Westmore rejected defendants' threshold contention that no discovery protocol was required. The court noted that defendants had previously offered a sampling protocol during negotiations and failed to suggest or show that implementing their own proffered framework would impose any undue burden. Turning to the review threshold, the court adopted plaintiffs' 10% standard instead of defendants' proposed 15% cutoff. The court observed that defendants articulated no particular burden associated with a 10% threshold. Moreover, because prior hit rates remained below 1%, few custodians were likely to fall into the 10% to 15% range, meaning defendants' two-tiered meet-and-confer proposal would unnecessarily increase the risk of collateral discovery disputes.
The court denied plaintiffs' demand for a null set sample of non-responsive, non-privileged documents. Plaintiffs argued that diverging views on relevance warranted validation sampling, citing prior disputes over match and win rates and production discrepancies with third-party productions. The court found that plaintiffs failed to show defendants withheld documents on relevance grounds after the earlier dispute was resolved, and offered no evidence that third-party discrepancies stemmed from non-responsiveness determinations. Furthermore, the court explained that case law authorizing null set sampling involved training sets for Technology Assisted Review models rather than statistical keyword search protocols, meaning plaintiffs failed to demonstrate a need for a null set.
Finally, the court denied plaintiffs' requests for a 2.5% margin of error and disclosure of recall rates, noting that plaintiffs offered no explanation justifying either requirement. The court accordingly adopted defendants' proposed protocol as modified: defendants must review random samples from each of the forty-seven custodians drawn to support a 95% confidence level with a 5% margin of error, and for any custodian with an estimated responsiveness rate of 10% or higher, defendants must review and produce all custodial documents containing plaintiffs' proposed search terms.
Key Takeaways
- A party resisting a discovery sampling protocol after having previously proposed one must demonstrate undue burden rather than simply asserting that additional discovery is unnecessary.
- Courts favor bright-line responsiveness thresholds over two-tiered negotiation triggers when low historical hit rates indicate that intermediate meet-and-confer processes would merely multiply discovery disputes.
- Null set validation sampling is primarily an element of Technology Assisted Review workflows and will not be imported into keyword search protocols without evidence of improper withholding.
- Parties seeking heightened sampling parameters, such as a reduced margin of error or recall rate disclosures, must provide affirmative justification showing why standard sampling metrics are insufficient.
Why It Matters
This decision clarifies that federal courts in the Northern District of California distinguish keyword search sampling from machine-learning Technology Assisted Review, declining to impose burdensome validation mechanisms like null set sampling without specific evidence of non-responsiveness withholding.