The Carpenter's Level and the Pendulum: Two Measures of Truth

There’s a quiet debate in the world of service reliability, one that plays out not in conference halls but in the configuration files of our monitoring tools. It’s a debate between two philosophies of measurement, embodied by two simple tools: the carpenter’s level and the pendulum.

The level represents the world of synthetic monitoring. We, the builders, set up a predefined check—a simple HTTP request, a database ping, a transaction script. We lay this check against our service like a level on a shelf, asking one binary question: is it plumb? Is it up? It gives us a clean, immediate, and absolute answer. The bubble is either centered or it is not. This is the world of uptime percentages, of clear success and failure. It is the tool of the architect, ensuring the foundation is sound.

The pendulum, however, belongs to the realm of observability. It doesn’t ask a predefined question; it simply observes the natural state of things. It is the continuous stream of metrics, logs, and traces—the real traffic from real users. A pendulum doesn’t tell you if something is level; it tells you about the forces acting upon it. Its swing reveals the subtle gravity of a slowing database connection, the atmospheric pressure of rising memory usage, the tremors of a network hiccup affecting a subset of users. Its truth is not absolute but relative, a story told in motion over time.

We often feel compelled to choose one camp. The pragmatist champions the level for its clarity and actionability. The detective advocates for the pendulum, valuing its rich narrative and emergent truths. But this is a false choice. A carpenter uses both. They use the level to confirm the initial structure is true, but they also watch how a door swings on its hinges—a natural pendulum—to diagnose if the frame has settled correctly under its own weight.

Likewise, a reliable service needs both truths. The synthetic check is our level, our contract that the basic plumbing works. It’s our first alert that something is catastrophically wrong. But the metrics from live traffic are our pendulum, showing us the smaller shifts, the gradual warping, the realities of performance that our perfect little test script could never anticipate. One gives us the stark fact of being online; the other tells us what it actually feels like to use the service.

Relying solely on the level leaves us blind to the slow drifts that precede a collapse. Trusting only the pendulum can overwhelm us with data, making it hard to see the simple, obvious break. The art is not in choosing one tool, but in learning to read them together—to understand the precise measurement of the level in the context of the swinging pendulum’s story.

Notes & further reading

A few pages I came back to while writing this: