The Penitence of the Pager: What My Grandfather's Broken Clock Taught Me About MTTR

My grandfather’s workshop smelled of oiled wood and ozone. On the wall, a large, ornate regulator clock, a family heirloom, hung in a state of perpetual silence. Its hands were frozen at 4:17. As a child, I asked him why he didn't fix it. He was a master craftsman, after all. He’d just smile and say, "Some silences are more honest than the wrong time." I didn't understand it then. Now, after years of being woken by pagers in the dead of night, I think I’m starting to.

In our world of uptime monitoring, we are obsessed with the illusion of motion. Green checkmarks, scrolling metrics, heartbeats that thrum with reassuring regularity. We want all the hands on our dashboard clocks to be moving, all the time. But my grandfather’s broken clock offered a different kind of truth. It wasn’t pretending. Its failure was absolute, quiet, and undeniable. It had reached a terminal state and declared it openly. There was no ambiguity, no flapping, no intermittent ticking that lied about its health. It was a clean, dignified failure.

Our services, of course, cannot simply stop. But in our rush to avoid the frozen hands, we often create a more insidious problem: the clock that ticks incorrectly. This is the world of the partial outage, the flickering indicator, the service that responds with a 200 OK but serves corrupted data. It’s a clock that says it’s 3:05 when it’s actually midnight. It’s a liar, and it sends us on wild goose chases, convincing us the problem is downstream or upstream, anywhere but itself. This is where the real penitence begins—not in fixing the broken, but in diagnosing the deceitful.

The pager’s alert is a call to penance. It’s the moment we are forced to confront the gap between the story our metrics tell and the reality our users experience. Mean Time To Resolution isn't just a business metric; it's a measure of our humility. The first step isn’t to restart the service blindly, hoping the hands will move again. It’s to approach the silent clock and truly listen. It’s to acknowledge the silence, to understand its cause, and to perform the repair with reverence for the truth it has revealed.

My grandfather eventually did fix the clock, but only after he had sourced the correct, historically appropriate movement. He didn’t just slap a quartz mechanism behind the face. He respected the integrity of the whole system. In our work, the equivalent is a thoughtful post-mortem, a robust fix, and a monitoring check that doesn’t just ask, "Is it alive?" but "Is it telling the truth?" The goal isn't to avoid the pager's wake-up call forever. It's to ensure that when it does sound, it’s for a honest, complete silence worth getting out of bed for, not for the frantic, dishonest ticking of a system that lost its way. The penitence is in the learning, and in the quiet satisfaction of restoring not just function, but truth.

Notes & further reading

A few pages I came back to while writing this: