Hugging Face Models on Foundry Managed Compute

This post details the integration of Hugging Face models into Microsoft Foundry's Managed Compute platform. It describes the curation pipeline for selecting, screening, and publishing models, including security checks and the use of SafeTensors. It highlights the various inference runtimes (vLLM, SGLang, TensorRT-LLM, NIM, TEI, llama.cpp) used to serve these models and explains how Foundry handles GPU management, container updates, and runtime upgrades automatically. The post emphasizes the unified developer experience across different deployment options (pay-per-token, provisioned throughput, Managed Compute) and the benefits of using open-weight models on Foundry, such as deep customization, self-hosting, and cost shaping.