TechPrufer
Sign inGet started

Blogs›Datadog›Kubernetes and AWS Networking Debugging

Kubernetes and AWS Networking Debugging

Kubernetes and AWS Networking Debugging

1
posts
2022

This post details a deep dive into debugging a complex incident involving gRPC, Kubernetes, and AWS networking. It highlights how issues that initially appear to be DNS-related can stem from deeper networking problems, including connection tracking, reverse path filtering, and the interaction between different networking layers (Cilium, Kubernetes, AWS). The post emphasizes the importance of comprehensive visibility through tools like ENA metrics and VPC Flow Logs for diagnosing such issues.

2022

It’s always DNS . . . except when it’s not: A deep dive through gRPC, Kubernetes, and AWS networking | Datadog

4/13/2022

This post details the investigation and resolution of an incident where DNS errors were initially suspected but ultimately traced to deeper networking issues within a Kubernetes environment on AWS. It covers the analysis of NodeLocal DNSCache behavior, including concurrency limits and health check failures, and the correlation of these issues with increased TCP retransmits. The post also explores the interaction between Cilium, Kubernetes, and AWS networking, and the impact of reverse path filtering on packet routing.