Your GPU Neighbor Can Hear Your Model — Branislav Brzak, Stealthium

Branislav Brzak demonstrates a timing side-channel attack on H100 GPUs that reveals information about a co-tenant's workload without accessing their memory. The attack leverages subtle latency differences (as small as three microseconds) caused by hardware operations affecting the entire GPU. By analyzing token rates and layer signatures derived from timing, the attacker can infer model depth and distinguish between workload categories like code, prose, JSON, and retrieval. The attack's effectiveness is shown to be influenced by speculative decoding patterns and can even be detected across NVLink. The talk concludes by discussing operator choices for mitigating such risks, including tenant placement and understanding isolation tradeoffs.