The Fallacy of the Single Health Check
We all do it. It’s DevOps 101. You spin up a new service, and one of the first tasks is to define a health check endpoint. You map it to `/health`, have it return a simple JSON blob with a `{"status": "OK"}` and a 200 status code. You plug that endpoint into your load balancer or orchestration tool, and you relax, confident that your monitoring has a solid foundation. The service is now "observable." But is it really? I’d argue that this single-point health check, while convenient, is one of the most seductive and dangerous fallacies in our quest for reliability.
The problem isn't the existence of a health check; the problem is the faith we place in it. We start to treat it as a binary oracle: if `/health` returns 200, the service is up; if it returns 503, it’s down. This binary view creates a false sense of security. It encourages a system design where an application can be "healthy" but utterly useless. Think about it: your service might pass its internal health check—database connection pool is fine, memory usage is normal—while a critical third-party API it depends on is timing out, or while a bug in a specific code path is causing 500 errors for a subset of users. The orchestrator sees green, but your users see red.
A Healthy Process vs. a Healthy Service
This confusion stems from conflating the health of the application *process* with the health of the service *functionality*. The basic `/health` endpoint is fantastic for the former. It tells the platform that the binary is running, it’s not crashing, and its immediate dependencies (like a local database) are reachable. This is crucial for automated recovery. If the process is sick, the platform should kill it and start a new one. But this is a low bar for what we consider "operational."
True service health is about capability, not just process state. Is the service able to fulfill its core purpose? Can it successfully complete the specific transactions that users and other services rely on? Answering this requires a more nuanced approach. It requires synthetic transactions or "deep" health checks that actively mimic user behavior. Instead of just checking for a database connection, a deep check would perform a small, safe, representative read and write. Instead of assuming a downstream service is fine, it would call a non-operational endpoint designed for validation.
Relying solely on the single, shallow health check is like judging a car’s roadworthiness only by checking if the engine starts. It’s a necessary first step, but it tells you nothing about the flat tire, the empty gas tank, or the broken brake light. You wouldn’t declare the car fit for a cross-country trip based on that one datum, yet we make analogous decisions about our services every day.
So, what’s the path forward? It’s not about abandoning the simple health check—it’s essential for orchestration. It’s about layering. The `/health` endpoint is for the platform. We then need a separate, more sophisticated system of checks—perhaps mapped to `/readiness` or `/deep-health`—that is used for a higher order of monitoring and decision-making. This is what truly informs you if a user’s request is likely to succeed. By clearly separating these concerns, we move beyond the fallacious binary of "up" or "down" and towards a more honest, functional understanding of what it means for our services to be truly reliable.
Notes & further reading
A few pages I came back to while writing this: