TensorRT-LLM

27 posts · 3 sources

NVIDIABlog2w

Enabling Private High-Performance Production AI Inference with NVIDIA Confidential Computing

NVIDIABlog2 Sept

Co-Designing AI Models Using Speculative Decoding for Faster LLM Inference

NVIDIABlog12 Aug

Serve Qwen3.8-2.4T-A95B, a 2.4T-Parameter Model, with Configurable Reasoning on NVIDIA GB300 NVL72

NVIDIABlog1 Aug

Co-Designing AI Model Attention for Fast, Interactive Long-Context Inference

NVIDIABlog23 Jun

Boost Inference Performance up to 15x on NVIDIA Blackwell Using DFlash Speculative Decoding

NVIDIABlog29 May

Run Step 3.7 Flash on NVIDIA GPUs with Enterprise-Ready Multimodal AI

NVIDIABlog27 May

NVIDIA Blackwell Sets STAC-AI Record for LLM Inference in Finance

BasetenBlog5 Feb

How to run LLM performance benchmarks (and why you should)

BasetenBlog3 Feb

The Baseten Inference Stack at NVIDIA Dynamo Day

BasetenBlog23 Jan

Open-sourcing Baseten’s suffix automaton MTP accelerator

BasetenBlog24 Oct

How we made the fastest GPT-OSS on NVIDIA GPUs 60% faster

BasetenBlog7 Aug

How we run GPT OSS 120B at 500+ tokens per second on NVIDIA GPUs

AtlassianBlog22 Jul

Atlassian’s Inference Engine, our self-hosted AI inference service

BasetenBlog5 May

Canopy Labs selects Baseten as preferred inference provider for Orpheus TTS models

BasetenBlog18 Apr

Accelerating inference with NVIDIA B200 GPUs

BasetenBlog27 Mar

How we built BEI: high-throughput embedding, reranker, and classifier inference

BasetenBlog27 Mar

Introducing Baseten Embeddings Inference: The fastest embeddings solution available

BasetenBlog9 Jan

Driving model performance optimization: 2024 highlights

BasetenBlog19 Dec

How we built production-ready speculative decoding with TensorRT-LLM

BasetenBlog19 Dec

Introducing our Speculative Decoding Engine Builder integration for ultra-low-latency LLM inference