Baseten

Introducing Custom Servers: Deploy production-ready model servers from Docker images

Dec 5, 2024
Introducing Custom Servers: Deploy production-ready model servers from Docker images
SummaryAI, from the source post

Introduced Custom Servers, a new feature that allows deploying model servers directly from Docker images using a YAML configuration. This bypasses the need for Truss Server for pre-packaged images (e.g., vLLM, Triton) or proprietary Docker images. Key features include a codeless developer experience, full integration with Baseten's ML infrastructure (autoscaling, GPU availability), and support for custom readiness and liveness probes. This extends the flexibility of Baseten's model deployment capabilities.

More from the Baseten AI Model Deployment and Serving Platform trail