TechPrufer
Get started
Datadog
Datadog logoDatadog · Feature trail

.NET Memory Profiling

4 posts · Jan 2024 – May 2024

What this trail covers

Datadog's .NET profiler has evolved to include memory usage profiling, enabling users to identify high CPU consumption from excessive garbage collection, pinpoint code responsible for memory allocations, and detect potential memory leaks by tracking surviving objects. This involves monitoring GC activity via CLR events, sampling allocations using the `AllocationTick` event, and tracking object lifetimes using weak handles and the `ICorProfilerInfo13` API. The system also addresses upscaling challenges by estimating real values based on sampled allocations and their sizes.

All Datadog trails
  1. .NET Continuous Profiler: Memory usage | DatadogThis post details the implementation of memory usage profiling in Datadog's .NET profiler. It covers monitoring garbage collector CPU impact by observing .NET Server GC threads, tracking allocations using the `AllocationTick` event (sampling every 100 KB), and monitoring surviving objects by leveraging `Weak` handles and the `ICorProfilerInfo13` API. It also addresses the challenge of upscaling sampled memory data by using size instead of duration, similar to lock contention profiling.
  2. .NET Continuous Profiler: Exception and lock contention | DatadogThis post details the implementation of exception and lock contention profiling within Datadog's .NET continuous profiler. For exceptions, it describes how to capture exception type, thread ID, and message by leveraging `ICorProfilerCallback::ExceptionThrown` and accessing internal exception fields like `_message`. For lock contention, it explains how to use CLR events (`ContentionStart`, `ContentionStop`) and the `EventPipeEventDelivered` callback to track lock contention duration and identify the holding thread, with specific considerations for different .NET runtime versions.
  3. .NET Continuous Profiler: CPU and wall time profiling | DatadogThis post details the implementation of CPU and wall time profiling within the Datadog .NET continuous profiler. It explains how application threads are monitored and sampled for CPU and wall time, including the use of `ICorProfilerCallback` methods for thread lifecycle management and the `StackSamplerLoop` class for sampling. It also covers the identification and monitoring of special native threads like those from the garbage collector. For CPU profiling, it describes the sampling frequency, the `IsRunning` function for checking thread status and CPU consumption on Windows and Linux, and the calculation of CPU time consumed. An internal example of using the CPU profiler to reduce CPU consumption is mentioned, along with performance optimizations for the `OsSpecificApi::GetCpuInfo` function.
  4. .NET Continuous Profiler: Under the hood | DatadogThis post details the technical architecture and implementation choices behind Datadog's .NET profiler, which is designed for continuous operation in production. It covers the profiler's components (samplers, providers, aggregator, exporter), how samples are collected and aggregated into the .pprof format using Rust, and how profiles are associated with traces and spans on the backend using runtime IDs. It also explains the process of cleaning up .NET call stacks for better readability and discusses the decision to implement the profiler in native code (C++ and Rust) running in the same process as the profiled application, rather than using Microsoft's TraceEvent nuget.