Meet Inkling: Thinking Machines Lab's new customizable model

This post details the deployment of Thinking Machines Lab's Inkling model (975B parameters, multimodal, MoE architecture with 41B active parameters and a 1M token context window) on the Baseten Platform. It highlights how Baseten's Inference Stack, including autoscaling and the Baseten Delivery Network (BDN), enables fast cold starts (2-3x faster) for large models. It also emphasizes the role of Multi-cloud Capacity Management (MCM) in ensuring availability and low latency by pooling GPUs across 20+ clouds, allowing workloads to shift to alternative clouds if capacity is constrained. The post also mentions the integration with vLLM for optimized serving.