Efficient Table Pre-training without Real Data: An Introduction to TAPEX

Introduced TAPEX (Table Pre-training via Execution), a novel method for pre-training language models on tabular data without requiring real-world data. TAPEX synthesizes a pre-training corpus by sampling executable SQL queries and their execution results over tables, effectively learning a neural SQL executor. This approach significantly improves pre-training efficiency and achieves new state-of-the-art results on table question answering and fact verification benchmarks, outperforming previous methods like TAPAS and TaBERT.