A/B Testing for LLM Endpoints

Together AI now offers A/B testing for LLM endpoints, allowing users to split live traffic into cohorts to measure candidate model performance with real users. This feature enables controlled ramp-ups, easy promotion of winning models via blue-green rollouts, and clean deletion of experiments. It integrates with existing platform metrics for comprehensive analysis and provides sticky cohorts per user if a sampling key is provided. The platform also supports multi-way tests with up to 20 variants and ensures consistent traffic shares independent of replica counts. Updates are guarded by etags for concurrent ramp safety, and the system handles experiment deletion gracefully, returning traffic to the base split.