Optimizing RTC bandwidth estimation with machine learning

This post details Meta's adoption of a machine learning (ML)-based approach to optimize Real-Time Communication (RTC) bandwidth estimation (BWE) and congestion control. It describes the transition from a parameter-tuned WebRTC module to an ML model that characterizes network types (random packet loss vs. congestion) and predicts future network conditions. The architecture combines LSTM for time-series data and dense layers for non-time-series data. The post shares experiment results demonstrating improvements in connection drop rate, video freeze percentage, and overall user experience across different network conditions, particularly for low-bandwidth users. It also outlines the challenges and learnings in generating training samples and deploying these models.