California Case Summaries

AMI – Government Employees Provident Fund Management Company Ltd., et al. v. Alphabet Inc., et al. — Adopting 10% Responsiveness Threshold for Custodial Document Search Protocol and Denying Null Set Validation

Unreported / Non-Citable

Case
AMI – GOVERNMENT EMPLOYEES PROVIDENT FUND MANAGEMENT COMPANY LTD., et al. v. ALPHABET INC., et al.
Court
U.S. District Court — Northern District of California
Judge
Kandis A. Westmore (District judges of the U.S. District Court for the Northern District of California, 2012)
Date Decided
2026-11-17
Docket No.
3:23-cv-01186
Status
Unreported
Topics
electronic discovery, search protocol, statistical sampling, null set validation, responsiveness threshold, custodian review, Securities Exchange Act

Background

Plaintiffs brought this securities class action in the Northern District of California against Alphabet Inc., Sundar Pichai, and Google LLC, alleging that defendants violated the Securities Exchange Act by making misleading statements concerning Google's digital advertising technology business. During discovery, the parties reached an impasse over the appropriate protocol for reviewing and producing custodial documents collected from two groups: twenty-seven custodians whose files had previously been produced in related ad tech antitrust litigation underlying plaintiffs' suit, and twenty newly added custodians. Earlier statistical sampling using plaintiffs' initial search terms generated low hit rates of 0.25% across the twenty-seven prior custodians and 0.06% across the twenty new custodians.

Plaintiffs subsequently demanded that defendants conduct statistical sampling on an individual custodian basis using revised search terms. During meet-and-confer discussions, defendants initially agreed to draw random samples at a 95% confidence level with a 5% margin of error across each of the forty-seven custodians, amounting to approximately 400 documents per custodian and nearly 20,000 documents in total. However, in the joint discovery letter submitted to the court, defendants changed position and argued that no protocol was necessary because further random sampling was unwarranted.

In the alternative, the parties presented competing proposals regarding the operational rules of any sampling protocol. Plaintiffs requested a 10% responsiveness threshold triggering document review and production, a null set sample of non-responsive documents to test categorization accuracy, a narrower 2.5% margin of error, and disclosure of recall rates. Defendants countered that any automatic review threshold should be set at 15%, with custodians yielding between 10% and 15% responsiveness reserved for further meet-and-confer discussions. Defendants also opposed producing a null set, arguing that null set validation is limited to technology-assisted review rather than keyword searches.

The Court’s Holding

United States Magistrate Judge Kandis A. Westmore rejected defendants' threshold contention that no discovery protocol was required. The court noted that defendants had previously offered a sampling protocol during negotiations and failed to suggest or show that implementing their own proffered framework would impose any undue burden. Turning to the review threshold, the court adopted plaintiffs' 10% standard instead of defendants' proposed 15% cutoff. The court observed that defendants articulated no particular burden associated with a 10% threshold. Moreover, because prior hit rates remained below 1%, few custodians were likely to fall into the 10% to 15% range, meaning defendants' two-tiered meet-and-confer proposal would unnecessarily increase the risk of collateral discovery disputes.

The court denied plaintiffs' demand for a null set sample of non-responsive, non-privileged documents. Plaintiffs argued that diverging views on relevance warranted validation sampling, citing prior disputes over match and win rates and production discrepancies with third-party productions. The court found that plaintiffs failed to show defendants withheld documents on relevance grounds after the earlier dispute was resolved, and offered no evidence that third-party discrepancies stemmed from non-responsiveness determinations. Furthermore, the court explained that case law authorizing null set sampling involved training sets for Technology Assisted Review models rather than statistical keyword search protocols, meaning plaintiffs failed to demonstrate a need for a null set.

Finally, the court denied plaintiffs' requests for a 2.5% margin of error and disclosure of recall rates, noting that plaintiffs offered no explanation justifying either requirement. The court accordingly adopted defendants' proposed protocol as modified: defendants must review random samples from each of the forty-seven custodians drawn to support a 95% confidence level with a 5% margin of error, and for any custodian with an estimated responsiveness rate of 10% or higher, defendants must review and produce all custodial documents containing plaintiffs' proposed search terms.

Key Takeaways

  • A party resisting a discovery sampling protocol after having previously proposed one must demonstrate undue burden rather than simply asserting that additional discovery is unnecessary.
  • Courts favor bright-line responsiveness thresholds over two-tiered negotiation triggers when low historical hit rates indicate that intermediate meet-and-confer processes would merely multiply discovery disputes.
  • Null set validation sampling is primarily an element of Technology Assisted Review workflows and will not be imported into keyword search protocols without evidence of improper withholding.
  • Parties seeking heightened sampling parameters, such as a reduced margin of error or recall rate disclosures, must provide affirmative justification showing why standard sampling metrics are insufficient.

Why It Matters

This decision clarifies that federal courts in the Northern District of California distinguish keyword search sampling from machine-learning Technology Assisted Review, declining to impose burdensome validation mechanisms like null set sampling without specific evidence of non-responsiveness withholding.

Read the full opinion (PDF) · Court docket

Scroll to Top