Real network mysteries, and how they were solved.

Each case links to the engineer's own write-up. Our short summary gives the symptom and how the investigation began. The answer is left for the original.

Each dot is a case, placed by year and by the layer where the root cause sat. Choose a layer to filter the list.

Cases

3 cases

Random stalls of 100 ms or more on connections to Kubernetes nodes

Debugging network stalls on Kubernetes, The GitHub Blog, 2019

Added 13 Nov 2026, sent in by tristan

Services on GitHub's Kubernetes clusters saw occasional delays of 100 ms or more, even for database queries that the database itself finished in milliseconds. The team stripped the path down one layer at a time: a load-test client, then bare TCP SYN probes with hping3, then the tunnel between nodes on its own, then plain ICMP between hosts. The delay survived every simplification, and the timings showed the stalls arriving in groups. The rest of the investigation follows that clue into the node itself.

kubernetestcpdumphping3bcclatency

A department couldn't send email farther than about 500 miles

The case of the 500-mile email, ibiblio.org, 2002

Added 13 Nov 2026

The chairman of a university statistics department called to say that email could not be sent farther than about 500 miles, and that his staff had drawn a map to prove it. The administrator was sure email does not work that way. Then he sent test messages to servers at different distances and found the same boundary. The story follows how he worked out why distance could matter to a mail server, down to a calculation that explains the number itself.

emailsmtpsysadmin

A Wi-Fi link only worked while it was raining

The Wi-Fi only works when it's raining, predr.ag, 2024

Added 13 Nov 2026

A home internet connection ran over a point-to-point Wi-Fi bridge. In dry weather it dropped almost every packet, but within minutes of rain starting it worked normally, and about a quarter of an hour after the rain stopped it failed again. The author pings each hop to find where the loss begins, tests the equipment over a wired connection, and checks cables, power supplies and antennas one by one. The explanation for why rain helped is not what anyone would guess first.

wifipingpacket-loss