The Voltmeter's Deception: On the Myth of the Empty Check
We’ve all written the kind of health check that’s little more than ceremony. It’s a reflex. A new service is deployed, and almost as an afterthought, we add the obligatory /health endpoint. It returns a simple, almost apologetic, 200 OK with a payload of {"status": "up"}. We wire it into the load balancer, pat ourselves on the back, and consider the service ‘monitored’. The circuit, we assure ourselves, is closed. But is the light we see truly an indicator of a functioning system, or just a bulb connected to its own private battery?
This is the deception of the empty check. It measures nothing but its own existence. It’s the systems administration equivalent of checking if a fuse is blown by seeing if the little ‘power on’ light on the fuse tester itself is illuminated. It tells you the tester has power, but reveals nothing about the complex, humming machinery downstream. The endpoint is alive, but is the service? Can it connect to its database? Is it slowly leaking memory? Are its third-party API dependencies responding, or has it entered a degraded state where it can still acknowledge a ping but can no longer perform its core duty?
We fall for this deception because it’s easy. A meaningful health check is hard work. It requires introspection. It forces the service to interrogate its own components, to attempt a miniature, sanitized version of its real work. A health check that does this—that verifies database connectivity, checks a cache, perhaps even validates a sample calculation—carries a cost. It consumes resources and, if designed poorly, could itself become a source of failure. The empty check is cost-free, risk-free, and, ultimately, value-free.
The Comfort of the Green Light
There’s a profound psychological comfort in that green light on a dashboard. It provides a false sense of security, a tidy checkbox that allows us to move on to the next task. The problem is that this comfort is a liability. It anaesthetizes our operational instincts. When an alert finally does fire, it’s not a gentle warning of a flickering light; it’s a five-alarm fire because the service has collapsed entirely. The empty check didn’t give us a gradual degradation signal; it lied to us right up until the moment the whole edifice crumbled.
A truly observant system doesn’t just answer ‘yes’ or ‘no’. It speaks about its well-being. It reports on the state of its vital organs. The shift we need to make is from mere liveness to genuine readiness. Liveness is binary: the process is running. Readiness is nuanced: the process is running and capable of handling work. This distinction is critical. It’s the difference between a car that has its engine on and a car that is in gear, has fuel, and is ready to drive.
Abandoning the empty check is an act of operational maturity. It’s an acknowledgement that reliability isn’t a boolean flag but a spectrum of states. Our monitoring should reflect this complexity. Instead of a single light, we need an instrument panel—one that shows database pool saturation, latency percentiles, and error rates alongside a simple ‘up’ indicator. We must build checks that are not just signals of life, but diagnostics of health. Otherwise, we are not engineers monitoring a system; we are spectators admiring a light bulb, oblivious to the darkness gathering just beyond its feeble glow.
Notes & further reading
A few pages I came back to while writing this:
- Chattanooga, TN
- The Unsent Postcard: Creating a Health Check That Acknowledges Reality
- Memphis, TN
- The Delicate Art of Not Looking
- Nashville, TN
- The Telegraph Clerk's Unread Message: A Precedent for Idempotency
- Amarillo, TX
- Austin, TX
- Brownsville, TX
- Carrollton, TX
- Corpus Christi, TX
- Dallas, TX
- Fort Worth, TX