TechPrufer
Get started
NVIDIA
NVIDIA logoNVIDIA · Feature trail

Hardware-Rooted AI Security with Confidential Computing

2 posts · Jul 2026 – Sep 2026

What this trail covers

This feature thread tracks the evolution of hardware-rooted AI security, focusing on NVIDIA Confidential Computing (CC) to protect data, code, and model integrity during inference with minimal performance overhead. Initial efforts focused on the challenges of securing AI workloads, leading to the development of CC integrated into Blackwell GPUs. Subsequent developments have demonstrated near-native inference performance (up to 98%) through optimizations in secure work submission, bandwidth management, and framework adaptations in TensorRT LLM for host-to-device data movement, kernel autotuner timing stabilization, and CC-aware multi-GPU communication. This enables running production AI inference on trusted hardware with minimal performance impact.

All NVIDIA trails
  1. Enabling Private High-Performance Production AI Inference with NVIDIA Confidential Computing | NVIDIA Technical BlogThis post details the performance impact of NVIDIA Confidential Computing (CC) on TensorRT LLM inference using NVIDIA Blackwell GPUs. It quantifies CC overhead by comparing CC-on and CC-off runs, showing retained throughput of 96.1-98.2% and TPOT overhead of 1.2-4.3%. It identifies specific CC-aware adaptations made in TensorRT LLM to mitigate performance degradation, including CC-aware memory selection for host-to-device transfers, asynchronous device-to-host readback, using the GPU %globaltimer for kernel autotuner timing, and choosing CC-aware multi-GPU communication algorithms. The post provides a methodology for measuring CC overhead on custom workloads.
  2. Hardware-Rooted AI Security That Won’t Slow You Down | NVIDIA Technical BlogThis post details the implementation and performance of NVIDIA Confidential Computing (CC) on Blackwell GPUs for AI inference. It highlights the hardware root of trust (fused private signing keys, NVLink encryption), the attestation process via NRAS, and performance optimizations in inference frameworks (FlashInfer, SGLang) to mitigate overheads. Benchmarks on HGX B300 with Qwen 3.5-397B-A17B-FP8 demonstrate that CC incurs minimal throughput and time per output token overhead (typically under 8%) compared to non-CC solutions, enabling secure, high-performance AI inference.