In high-frequency trading data, noise isn't the problem. Assumptions are.

This post introduces the problem of data preprocessing in high-frequency trading, highlighting that fixed normalization strategies fail due to non-stationary market conditions. It reframes the problem as a representation learning challenge and discusses existing research in limit order book (LOB) representation learning. The core contribution is the exploration of a Bayesian framework to learn market structure directly from historical order book data, enabling adaptive preprocessing. This approach uses statistical divergence to distinguish between noisy and novel data, aiming to reduce training inefficiency and improve GPU utilization. The post emphasizes the practical implications for quantitative firms and Lambda's role in bridging infrastructure and applied ML fluency.