TechPrufer
Get started
Together AI
Together AI logoTogether AI · Feature trail

Open Source AI Stack

2 posts · Aug 2025 – Sep 2026

What this trail covers

The Together AI platform supports a wider range of open-weight models, including GLM-5.3 and GLM-5.3 Flash, with enhanced experiment tracking that provides live metrics for loss, gradient norm, and learning rate. New controls include expert LoRA for Mixture-of-Experts models, early stopping based on validation loss, and arbitrary batch size support via gradient accumulation. Data processing is more transparent with tokenized data inspection and the ability to download prepared datasets. This release also details a successful migration strategy from closed-source to open-source models, emphasizing a phased approach involving discovery, evaluation, adaptation, decision-making, and production deployment. The strategy highlights the importance of defining use-cases, leveraging benchmarks, performing custom evaluations with real data, adapting prompts and inference settings, and communicating ROI to stakeholders. Managed services are presented as a way to accelerate this migration.

All Together AI trails
  1. Migrating from closed to open source models, TogetherThis post details a migration strategy from closed-source to open-source models, outlining a phased approach: discovery (defining use-cases, leveraging benchmarks, identifying model tiers), evaluation (accuracy vs. performance, using real data replay, setting goals), adaptation (system prompt engineering, inference settings, context engineering, fine-tuning/distillation), decision-making (packaging effort, risks, ROI), and production deployment (canary deployments, roadmap based on effort/risks). It emphasizes that managed services can significantly accelerate this migration.
  2. The Open Source AI StackThis post details the "MIGHT" stack (Model, Inference, Gateways/Routers, Harness, Tools) for utilizing open-source AI models, particularly for agentic software development. It explains the role of different model sizes (large vs. small) and their trade-offs, introduces cloud inference providers and gateways for model hosting and routing, and describes the harness as the application layer managing interaction with models and tools. It highlights the flexibility of choosing models independently from providers and the importance of treating models as tools in a developer's toolbox.
  3. GLM-5.3 vs. GLM-5.3 Flash on DeepSWE: Cost, Coding, and RoutingThis post details a comparative analysis of GLM-5.3 and its distilled variant, GLM-5.3 Flash, on the DeepSWE benchmark. It quantifies the trade-offs between capability, consistency, and cost. Key findings include that GLM-5.3 Flash sacrifices consistency and the ability to 'think its way to a win' on flaky tasks, rather than core capability. The cost reduction is substantial (17x), with Flash being faster per step. A cascade strategy of using Flash first and escalating to GLM-5.3 for difficult tasks is proposed, achieving 80.9% task completion at a significantly lower cost than using GLM-5.3 alone. A genuine regression in baseline test breakage rate for Flash is identified, suggesting the need for guardrails.
  4. Together AI brings Thinking Machines Lab’s new model Inkling on day 0This post announces the availability of Thinking Machines Lab's Inkling model on the Together AI platform. Inkling is a multimodal mixture-of-experts model with 975B total parameters and 40B active parameters per token, supporting text, image, and audio inputs with a 1M context window. It features query-conditioned relative attention, short causal convolutions, and a shared-sink MoE architecture. The post highlights Together AI's optimized FlashAttention-4-based kernel for efficient production inference of Inkling's unique attention mechanism and its multimodal input handling via a single API endpoint.
  5. Together AI Brings NVIDIA Nemotron 3 Nano Omni to Developers on Day 0This post announces the availability of NVIDIA Nemotron 3 Nano Omni on the Together AI platform. It highlights the model's multimodal reasoning capabilities across video, images, audio, and language, and its hybrid Mamba-Transformer MoE architecture. The post emphasizes Together AI's optimizations for high throughput, cost-efficient, and low-latency inference, as well as its managed infrastructure for agentic workloads and secure APIs.
  7. Together AI Brings NVIDIA Nemotron 3 to Developers on Day 0This post announces the availability of NVIDIA Nemotron 3 Super on Together AI's Dedicated Inference platform. It highlights Nemotron 3 Super's hybrid MoE architecture (Transformer + Mamba), 1M-token context window, and multi-token prediction capabilities, which are beneficial for agentic workflows and complex reasoning. The post details how Together AI's infrastructure, including the Together Inference Engine and custom CUDA kernels, optimizes the execution of this large model, enabling single-GPU deployment and production-grade isolation.
  8. How to choose the right open model for productionThis post provides guidance on selecting open models for production workloads. It details the benefits of open models (transparency, adaptability, control), legal considerations (licensing, origin), and compares open models to proprietary tiers based on parameter size and capabilities. It outlines trade-offs between cost, speed, and quality with concrete examples of model configurations. The post also emphasizes disciplined experimentation, LLM-as-a-judge evaluations, manual review for failure mode analysis, and fine-tuning techniques (SFT, DPO, LoRA) for task adaptation.
  9. Announcing native availability of NVIDIA Nemotron 3 Nano, NVIDIA’s latest reasoning modelIntroduces native availability of NVIDIA Nemotron 3 Nano, a hybrid Mamba-Transformer + sparse MoE architecture with ~3B active parameters. Highlights its optimization on Together AI for high throughput and cost-efficiency, and its suitability for agentic systems due to its 1M-token context window and open weights, data, and training recipes. Details its performance in coding, math, scientific reasoning, and function calling, and its benefits for production-scale reasoning and agentic workloads on Together AI, including performance, reliability, cost efficiency, and flexibility.
  10. Announcing the Availability of OpenAI's Open Models on Together AIThis post announces the availability of OpenAI's gpt-oss-120B and gpt-oss-20B models on Together AI's infrastructure. It highlights their Apache 2.0 licensing, enabling unrestricted modification and deployment. The models are offered via serverless and dedicated endpoints, emphasizing superior economics, faster inference through optimizations, transparent pricing, 99.9% uptime SLA, multi-region deployment, SOC 2 compliance, and zero throttling. The post details how Together AI's platform, in partnership with NVIDIA, provides research-grade tooling, fine-tuning capabilities, and OpenAI-compatible APIs for seamless migration. It also mentions specific pricing for input/output tokens and the Batch API for cost-effectiveness. Real-world applications in emerging markets, government digital sovereignty, research, and enterprise customization are discussed. A Python SDK example for accessing the models is provided.