TechPrufer
Get started
DeepMind
DeepMind logoDeepMind · Feature trail

LLM Factuality Benchmarking

1 post · Dec 2025

What this trail covers

The FACTS Benchmark Suite is introduced, extending previous work on factuality evaluation. It includes three new benchmarks: Parametric (measuring internal knowledge accuracy), Search (testing tool use for information retrieval and synthesis), and Multimodal (evaluating factually accurate responses to image-based prompts). The original FACTS Grounding benchmark has also been updated to v2. The suite comprises 3,513 examples, with a held-out private set managed by Kaggle. Gemini 3 Pro achieves a 68.8% overall FACTS score, with significant improvements in Search & Parametric slices.

All DeepMind trails
  1. FACTS Benchmark Suite: a new way to systematically evaluate LLMs factualityIntroduces the FACTS Benchmark Suite, comprising Parametric, Search, and Multimodal benchmarks, alongside an updated Grounding Benchmark v2. Details the methodology, dataset size (3,513 examples), and evaluation process managed by Kaggle. Presents results showing Gemini 3 Pro leading with a 68.8% FACTS Score, highlighting error rate reductions in Search and Parametric benchmarks. Notes that Multimodal benchmarks generally had lower scores across all evaluated models.