TechPrufer
Get started
Anthropic
Anthropic logoAnthropic · Feature trail

LLM Training Data Quality and Robustness

2 posts · May 2022 – Dec 2022

What this trail covers

Anthropic is investigating the impact of data repetition in LLM training. This post details findings on how repeated data, even in small fractions, can lead to significant performance degradation through a double descent phenomenon. The research connects this to mechanistic interpretability, showing that repetition damages generalization structures like induction heads, shifting the model towards memorization. This work aims to understand and mitigate the harms of data repetition to improve LLM generalization and robustness. This post extends the investigation to limited datasets and provides a hypothesis for why repeating a relatively small fraction of data in large language models could lead to disproportionately large harms to performance.

All Anthropic trails
  1. Tracing Model Outputs to the Training DataThis post introduces an efficient method for scaling influence functions to large language models (up to 52 billion parameters) to understand how training data influences model outputs. It observes that generalization patterns become more abstract with model scale and that influence is distributed across layers but can be localized. The research connects these findings to mechanistic interpretability, showing how data repetition can damage generalization structures.
  2. Scaling Laws and Interpretability of Learning from Repeated DataThis post systematically studies the effects of repeated data in LLM training. It finds a strong double descent phenomenon where repeated data can increase test loss midway through training. A predictable range of repetition frequency leads to severe performance degradation; for instance, repeating 0.1% of the data 100 times can degrade an 800M parameter model's performance to that of a 400M parameter model. The work connects these observations to mechanistic interpretability, showing that data repetition disproportionately damages copying and internal structures associated with generalization, such as induction heads, providing a possible mechanism for the shift from generalization to memorization.