Confidential Inference via Trusted Virtual Machines

This post details the technical approach to Confidential Inference using trusted virtual machines and confidential computing. It outlines the architecture involving a trusted loader program that runs within an isolated, hardware-secured environment (e.g., a confidential VM). This loader is responsible for decrypting encrypted data, invoking hardware accelerators, and re-encrypting results. The system relies on cryptographic proofs (attestation via TPMs) to verify the integrity and configuration of the loader, ensuring that only authorized and reviewed code can access decrypted data. The post also discusses the implications for both model weight security and user data privacy, and suggests hardware design considerations for future accelerators.