Trading Airflow + EMR for Temporal + Bauplan: The Mediaset tale

This post details Mediaset's migration from a legacy Airflow + EMR stack to Temporal and Bauplan. It highlights the challenges of the previous setup, including slow data refresh rates and complex infrastructure. The migration to Temporal for orchestration and Bauplan for serverless data pipelines resulted in a near real-time dashboard with a 5-minute refresh rate, reduced development cycles from 3 months to 6 weeks, and simplified infrastructure from over 6 AWS services to just 2 tools. The post emphasizes Temporal's advantages over Airflow, such as durable, replayable workflows with automatic retries and a code-first approach, and Bauplan's serverless data management capabilities.