The Cartographer's North Star: On the Deception of the Perfectly Centered Map
In the quiet, digital war rooms where we guard the ramparts of our services, a particular piece of wisdom is treated as gospel: the value of a single, definitive average. We calculate the average response time, the average uptime, the average packet loss. We plot these numbers in the center of our dashboards, believing they represent the true north of our system’s health. This average becomes our North Star, a fixed point by which we navigate. But what if this lodestar is a phantom, a comforting illusion that, in its simplicity, steers us toward the rocks?
The problem isn’t that the average is a lie, but that it is an erasure. It smudges the stark lines of reality into a gentle, misleading blur. Consider a service with an average latency of 200 milliseconds. This single number could describe a beautifully consistent performance where every user enjoys a 200ms experience. More often, however, it describes a scenario where ninety-five percent of users get a zippy 50ms response, while the remaining five percent suffer through a ponderous 3.5 seconds. The average is mathematically correct, but it is operationally dishonest. It silences the screams of that suffering minority, muffling their experience into a statistic that suggests everything is, on the whole, just fine.
This obsession with the center is a form of cartographic vanity. It’s like drawing a map of a continent and placing your own city squarely in the middle. It feels right to you, but it distorts the true relationships between all other points. By centering the ‘normal,’ we push the outliers—the very events that signal impending doom or systemic unfairness—to the ragged edges of our perception. We become masters of the common path and blind to the failure modes that matter most. The user connecting from a rural area, the customer on an older device, the database shard experiencing subtle memory pressure—their struggles are averaged out of existence.
True observability requires not a fixed point, but a dynamic understanding of the entire landscape. It demands we trade our compass for a detailed relief map. Instead of worshipping the average, we must learn to read the distributions, the percentiles (the 95th, the 99th), and the histograms. These tools don’t give us a single, easy answer. They show us the shape of the experience: the dense clusters, the long tails, the gaps and the outliers. They force us to ask not “How are we doing?” but “How is *everyone* doing?” and, more importantly, “Who is having the worst experience and why?”
The perfectly centered map is a seductive ideal. It offers a clean, simple narrative of control. But the world of distributed systems is messy, jagged, and full of hidden coves and treacherous shoals. Our guiding light shouldn’t be a single point that ignores the extremities; it should be a commitment to illuminating the entire terrain, especially the dark corners where the averages never dare to tread. Only then can we truly claim to know the lay of the land we are responsible for navigating.
Notes & further reading
A few pages I came back to while writing this: