TechPrufer
Sign inGet started

Blogs›Datadog›Datadog Agent Mount Handling

Datadog Agent Mount Handling

Datadog Agent Mount Handling

1
posts
2017

The Datadog Agent has evolved to handle potential system hangs caused by `os.statvfs` calls on NFS mounts. Initially, the agent would stall during disk checks when encountering problematic NFS mounts, leading to gaps in metrics. The system was improved by running `os.statvfs` on a separate thread and implementing a timeout mechanism to prevent the main agent thread from hanging. This ensures continued operation even with unreliable NFS connections, though it may slightly increase memory usage on affected systems.

2017

The trouble with mounting | Datadog

1/23/2017

This post details the investigation and resolution of a problem where the Datadog Agent would hang during disk checks due to `os.statvfs` calls on NFS mounts. The root cause was identified as `statvfs` hanging on hard-mounted NFS directories, exacerbated by glibc's implementation that iterates through `/proc/mounts`. The solution implemented was to run the `os.statvfs` call on a separate thread with a timeout, allowing the main agent thread to continue processing even if the `statvfs` call hangs.