Kroll Reduces First-Pass Review Document Population by 90% with Relativity aiR for Review

How did they do it?

• Leveraged generative AI to determine first-pass relevance decisions across 80,000 documents, with oversight from client subject matter experts and counsel
• Validated AI decisions against statistically significant human-coded samples
• Achieved 94.4% recall, ensuring nearly all relevant material was captured while minimizing false negatives
• Cut tens of thousands of manual review hours – turning a time-intensive review into a targeted, consistent, and defensible workflow

Delivering at Speed Without Compromising Precision

Dealing with a legal dispute is one of the most demanding challenges an organization can face, especially in today’s world of large data volumes and short timelines. A consumer-sector company felt that pain firsthand in a matter relating to alleged misuse of brand and technology assets and contractual payments.

The company had a large and diverse email data set of about 80,000 documents requiring initial review. An initial attempt at coding 22,000 documents demonstrated that manual first‑level review would be time‑consuming, costly, and difficult to scale.

Needing to balance precision and pace, the client was interested in using AI to perform full first‑pass review with expert-led quality control.

Combining Human Insight with AI Automation

With their goals in mind, the company’s legal team worked with their solution provider, Kroll – a Relativity Gold Provider Partner with 40+ Relativity certifications and 18+ Relativity Generative AI Pro-certified individuals within their team.

With efficient and accurate document analysis critical to the company’s legal action, Kroll knew which tool would deliver the best results within the very limited timeframe: Relativity aiR for Review.

Moving fast, Kroll undertook rapid prompt development, ingesting matter materials – including the Statement of Claim – and using aiR’s prompt kickstarter feature to generate initial prompts. Kroll next worked with the company to refine prompts with client‑driven terms, entities, and contextual criteria.

Kroll then conducted live workshops with the client to review conflict and borderline documents, tuning prompts in real time for accuracy and consistency. This was followed by leveraging stratified samples to ensure representation across clusters and topic areas for prompt testing.

The critical stage involved defensibility validation, during which a statistically valid sample, designed to achieve a 99% confidence level with a ±5% margin of error, was used to assess the tool’s performance against defined criteria. Building on this foundation, Kroll conducted an error‑matrix‑based performance assessment, followed by targeted re‑review of conflict documents, to confirm accuracy and coding consistency.

Surfacing the Evidence that Matters the Most

By leveraging aiR for Review, Kroll turned what would have been a time-intensive first-level review into a targeted, efficient, defensible workflow, with AI decisions validated against statistically significant human-coded samples – allowing their client’s legal teams to quickly surface the most valuable and impactful evidence.

After validation, aiR for Review demonstrated 94.4% recall, with aiR capturing nearly all relevant material. The required human review set was reduced to around 3,500 documents, of which around 1,000 were expected to be truly relevant.

The impact for Kroll’s client was an over 90% reduction in documents requiring full manual review, dramatically accelerating timelines and enabling counsel to focus on the most relevant evidence as early on as possible. The team also achieved cost savings proportional to the reduction of tens of thousands of manual review hours.

“Bringing human expertise together with Relativity aiR for Review meant that Kroll accelerated what would have been a time-intensive first-level review into a targeted, efficient workflow, dramatically reducing turnaround times and enhancing defensibility. By leveraging solutions at the forefront of document review technology, we ensure our clients gain the critical insights they need right when they need them most.”
Bailey Bates, Kroll, Discovery & Analytics Solutions