Upgrading one of the oldest components in Cloudflare’s software stack

This post details a custom NGINX upgrade mechanism for the FL proxy that avoids doubling memory requirements during releases. The mechanism involves disabling automatic worker restarts, shutting down one worker at a time from the old instance, starting a new worker for the new instance, and then re-enabling automatic restarts. This process is repeated for all workers, ensuring that memory usage remains stable and that CPU pinning is maintained throughout the upgrade. This custom upgrade mechanism enables reliable releases of large applications like FL regardless of available memory on edge servers.