Introducing Custom Servers: Deploy production-ready model servers from Docker images
Dec 5, 2024

Summary
Introduced Custom Servers, a new feature that allows deploying model servers directly from Docker images using a YAML configuration. This bypasses the need for Truss Server for pre-packaged images (e.g., vLLM, Triton) or proprietary Docker images. Key features include a codeless developer experience, full integration with Baseten's ML infrastructure (autoscaling, GPU availability), and support for custom readiness and liveness probes. This extends the flexibility of Baseten's model deployment capabilities.
More from the Baseten AI Model Deployment and Serving Platform trail
- LangChain trains custom models for LangSmith Engine with Baseten Loops
- DeepSeek-V4.1-Flash: more efficient prefill for coding agents
- Blaxel is joining Baseten to build the future of agentic infrastructure
- How Baseten makes pyannote’s diarization models 9.6x faster
- Optimizing delta weight syncs for managed rollouts