The Insistence of the Untrodden Path: Why Your Health Check Is a Liar

It reports a clear, steady green. ‘All systems operational,’ it chirps, a digital canary blissfully singing in a coal mine you haven’t entered yet. You’ve defined the perfect health check endpoint, a dedicated route that returns a 200 status code faster than a blink. It verifies database connectivity, checks a cache, and reports a happy, healthy state. You trust it. And in doing so, you may have been lulled into a dangerous complacency.

The problem is not that the health check is wrong, in its own limited way. It’s that it’s telling a truth about a path that no real user will ever take. It’s a custodian of a pristine, curated garden path, while your actual users are crashing through the untamed wilderness just beyond the hedges. Your service can be perfectly healthy on the route you’ve designated for inspection, while the critical user journey—the payment flow, the search query, the file upload—is quietly smoldering.

The Map and the Territory

This is the fundamental deception of synthetic monitoring: it confirms the existence of a path, but not the condition of the territory. A health check is a map you drew yourself, and it’s always accurate to its own specifications. But a user’s request is an explorer, not a cartographer. It doesn’t follow your neat lines; it forges its own way through the complex logic of your application, hitting code paths that are never exercised by your simple, deterministic check.

Think of it as checking the locks on your front door every hour. The lock is solid, the door is firm. Your ‘home security health check’ passes. But you’ve failed to notice that a window around the back has been left ajar, and rain is pooling on the floorboards. The core structure is sound, but the lived-in functionality of the house is compromised. Your health check verified the fortress’s gate, but it never walked the perimeter.

This is where the concepts of uptime and reliability violently diverge. Uptime is a binary state: is the process running? Can it respond to a basic request? Reliability, however, is the quality of the experience for a real person trying to accomplish a real task. A service can have 99.99% uptime and be utterly unreliable if that 0.01% of failure occurs precisely when a user is trying to complete a purchase, or if a latent bug causes a critical feature to fail silently.

The way out of this trap is to make your health checks less honest, and more like liars who have seen the wider world. Instead of a single, simple endpoint, you need a suite of checks that impersonate actual user behavior. Can a user sign in? Add an item to a cart? Submit a form? These are not trivial to implement; they require maintaining state, handling authentication, and navigating the same complex UI or API flows your users do. They are costly, both in resources and maintenance. But they are the only checks that tell you about the territory, not just the map.

Your service’s true health isn’t measured by the path you keep clear for inspection. It’s measured by the insistence of the untrodden path—the one a user discovers in the middle of the night, under a load you didn’t anticipate. Stop asking your system if it’s alive. Start asking it if it’s usable. The answer, you may find, is very different.

Notes & further reading

A few pages I came back to while writing this: