Skip to content
Network Case Files

Real network mysteries, and how they were solved.

Each case links to the engineer's own write-up. Our short summary gives the symptom and how the investigation began. The answer is left for the original.

Each dot is a case, placed by year and by the layer where the root cause sat. Choose a layer to filter the list.

Cases

1 case

Random stalls of 100 ms or more on connections to Kubernetes nodes

Debugging network stalls on Kubernetes, The GitHub Blog, 2019

Added 13 Nov 2026, sent in by tristan

Services on GitHub's Kubernetes clusters saw occasional delays of 100 ms or more, even for database queries that the database itself finished in milliseconds. The team stripped the path down one layer at a time: a load-test client, then bare TCP SYN probes with hping3, then the tunnel between nodes on its own, then plain ICMP between hosts. The delay survived every simplification, and the timings showed the stalls arriving in groups. The rest of the investigation follows that clue into the node itself.

kubernetestcpdumphping3bcclatency