Transparent memory offloading: more memory at a fraction of the cost and power

This post introduces Transparent Memory Offloading (TMO), Meta's data center solution for addressing the growing memory needs and high cost of DRAM. TMO utilizes cheaper, higher-capacity memory technologies like NVMe SSDs and compressed memory by transparently offloading less frequently accessed memory pages. It employs a new Linux kernel mechanism (PSI) to measure workload pressure and a userspace agent (Senpai) to dynamically adjust offloading, achieving significant memory savings (20-32%) across millions of servers and contributing to reduced cost and power consumption.