RLTL;DR: Self-Improvement by Internalizing Self-Generated Feedback

Introduced RLTL;DR, a self-improvement method for RL agents. Key components include: 1. Generating 'TL;DR' insights from verifier outputs after failed attempts. 2. Conditioning subsequent rollouts on these insights. 3. Enabling backpropagation on in-context insights to create a direct task-to-insight mapping. Demonstrated significant performance gains (Pass@1 of 14-31%) on challenging tool-calling and coding datasets where standard GRPO training yielded 0-1% Pass@1. Showcased that SFTL;DR, training solely on (task, insight) tuples, recovers almost full performance, highlighting the efficacy of internalized feedback.