How we analyzed and fixed a Golang memory leak

This post details the debugging and resolution of a Golang memory leak in a containerized App Engine flexible instance. The leak was identified as an interaction between Golang's MADV_FREE memory management and the container environment, where memory was not being returned to the OS. The solution involved setting the GODEBUG environment variable to "madvdontneed=1" to revert to MADV_DONTNEED behavior, which resolved the issue and stabilized memory usage. Key takeaways include the utility of pprof and continuous profilers for memory leak analysis, the use of MemStats for fragmentation estimation, and the recommendation to set GODEBUG="madvdontneed=1" for Go versions 1.12-1.15 running in containers.