Measuring the Persuasiveness of Language Models

This post details a new methodology for measuring the persuasiveness of language models, comparing different generations and classes of Anthropic models against human-written arguments. It establishes a scaling trend where newer, larger models are more persuasive, with Claude 3 Opus achieving human-level persuasiveness. The research also explores the implications of AI persuasiveness for potential misuse and releases experimental data.