Day zero benchmarks for Qwen 3 with SGLang on Baseten

This post details the day-zero deployment and benchmarking of Qwen 3 using the SGLang inference framework. It outlines the Mixture of Experts (MoE) architecture of Qwen 3 235B, explaining the sharding strategies for attention (Tensor Parallelism) and sparse MoE layers (Expert Parallelism). Performance benchmarks are provided for Qwen 3 235B with SGLang, showing median time-to-first-token, tokens per second per user, and total throughput. The post also discusses optimizations for cold start times for large model weights and offers inference tips for Qwen 3, including varying temperature, leveraging agentic capabilities, and utilizing the context window.