TechPrufer
Sign inGet started

Blogs›Together AI›Dedicated Container Inference

Dedicated Container Inference

Dedicated Container Inference

4
posts
2026

Together AI has significantly updated its inference platform, offering enhanced control over performance, cost, and quality for open-weight and licensed models. Key improvements include production-grade deployment features like canary, blue-green, and rolling updates with auto-rollback, A/B and shadow testing for real traffic evaluation, and advanced autoscaling capabilities. The platform now supports custom training betas for full-weight and LoRA reinforcement learning and supervised fine-tuning, with seamless deployment of trained checkpoints. Model caching and distribution have been rebuilt for faster warm starts (up to 4x). The platform provides an organization-level Prometheus endpoint for observability and an improved in-product analytics experience. This update aims to simplify the transition from experimentation to production, enabling continuous model improvement without requiring users to build their own inference stack.

2026

The production platform for open-weight AI inference

7/23/2026

This post introduces a major update to Together AI's inference platform, focusing on production-grade deployment capabilities for open-weight and licensed models. It details features for safe model rollouts (canary, blue-green, rolling updates with auto-rollback), traffic testing (A/B, shadow), and autoscaling based on various inference-native metrics. The platform now offers improved model caching for faster warm starts and an organization-level Prometheus endpoint for observability. Additionally, it announces a closed beta for custom training, including RL and SFT, with direct deployment of checkpoints to production inference endpoints.

Deploy and inference any model from HuggingFace

5/8/2026

This post details the process of deploying an arbitrary model (Netflix's void-model) from HuggingFace onto Together AI's Dedicated Container Inference (DCI) infrastructure using the Goose CLI agent and the `together-dedicated-containers` skill. It outlines the steps involved: installing the skill, running a single prompt to initiate the deployment, and then using the Together CLI to submit inference requests to the deployed model. The post highlights the benefits of DCI for rapid deployment of new models without extensive manual configuration.

Together AI Brings NVIDIA Nemotron 3 to Developers on Day 0

3/11/2026

This post announces the availability of NVIDIA Nemotron 3 Super on Together AI's Dedicated Inference platform. It details the model's hybrid MoE architecture (Transformer + Mamba), 1M-token context window, and multi-token prediction capabilities, highlighting their benefits for agentic workflows and complex reasoning. The post also explains how Nemotron 3 Super is optimized for single-GPU deployment on H200/H100 GPUs within Together AI's managed infrastructure, emphasizing the use of the Together Inference Engine and custom CUDA kernels for accelerated performance and production-grade isolation with an SLA and SOC 2 compliance.

Introducing Dedicated Container Inference: Delivering 2.6x faster inference for custom AI models

2/12/2026

Introduces Dedicated Container Inference, a new capability for deploying custom generative media models. This feature provides production-grade orchestration including autoscaling, queuing, traffic isolation, and monitoring for user-provided Docker containers. It supports job orchestration with independent queues, policy-driven traffic control, and isolation between different traffic types. The architecture treats containers as the unit of execution, utilizes volume mounts for model weights, and offers autoscaling based on queue depth or custom metrics. Observability is built-in.