A look inside the Cloudflare ML Ops platform

This post details Cloudflare's internal ML Ops platform for model training. Key contributions include: 1. Notebooks: Deployment of JupyterHub on Kubernetes for scalable and collaborative data exploration and model experimentation. 2. GitOps: Adoption of GitOps with ArgoCD for continuous delivery and infrastructure management, enabling declarative infrastructure and application code management via pull requests. 3. Templates: Introduction of production-ready model templates for training (ETL, feature engineering, experiment tracking, DAG orchestration), batch inference, stream inference (FastAPI microservices), and explainability (Streamlit/Bokeh dashboards). 4. Orchestration: Support for multiple open-source orchestration tools including Apache Airflow, Argo Workflows, Kubeflow Pipelines, and Temporal, allowing teams to choose based on use case and preference. 5. Hardware: Focus on optimizing GPU and CPU utilization for core datacenter workloads and edge inference.