How we wrote a Python profiler | Datadog

What this post added

This post details the development of Datadog's statistical Python profiler. It explains the rationale for statistical profiling over deterministic profiling (like cProfile) for production environments due to overhead concerns. The architecture is described, including the recorder, collectors (stack, memory, lock), exporter (pprof format), and scheduler. The stack collector's implementation in Cython and its dynamic polling rate adjustment for low overhead are highlighted. The memory and lock collectors are also introduced.