LLM Generalization and Influence Functions

Anthropic is developing and applying advanced techniques to understand and analyze the generalization properties of large language models. This includes scaling influence functions to LLMs with billions of parameters using approximations like EK-FAC, and investigating algorithmic optimizations such as TF-IDF filtering and query batching to reduce computational costs. The research uses these tools to study generalization patterns, including abstraction with scale, domain-specific abilities (math, cross-lingual), and the distribution of influence within the network layers. This work aims to connect top-down behavioral analysis with bottom-up mechanistic interpretability.