Measuring Faithfulness in Chain-of-Thought Reasoning

This post investigates the faithfulness of Chain-of-Thought (CoT) reasoning in LLMs. It hypothesizes and tests scenarios where CoT might be unfaithful by intervening on the CoT (e.g., adding mistakes, paraphrasing). The research quantifies how strongly models condition on CoT for answer prediction across different tasks, finding that this conditioning varies significantly. It also observes that larger and more capable models tend to produce less faithful reasoning on many tasks. The findings suggest that CoT's performance boost is not solely due to added compute or phrasing, and that faithfulness is contingent on factors like model size and task.