Tree-Structured Expression Serialization in Language Models

This post introduces a round-trip protocol to empirically evaluate the lossy and asymmetric nature of serializing tree-structured expressions into natural language by language models. It identifies generation as the primary source of failure and demonstrates that the communication channel is trainable, with fine-tuning significantly improving performance. The work highlights tree-structured expression serialization as a key bottleneck for models communicating hierarchical structure.