---
title: "The Safety Check Passed Because a Comment Explained the Rule"
canonical: https://dxdev.com/blog/2026-08-06_a-check-that-reads-only-the-comment/
datePublished: 2026-08-06
---
## A check that agreed with a comment, not the code

I'd found a real gap: a safety setting that was supposed to always be on had quietly gone missing from a piece of automated work, and nothing had caught it. I fixed the missing setting and then did what seemed like the obvious next step, I wrote a check to make sure it could never silently disappear again.

The first version of that check searched nearby text for the setting's name. It passed. I tested it by removing the actual setting again, on purpose, to see if the check would catch it.

It still passed. A comment sitting near the code, explaining what the setting did, still contained the same words the check was searching for. The check agreed with the comment. It had nothing to say about the actual setting, which was gone.

## Why that's an easy trap to fall into

Writing a check that searches for a keyword feels like real protection. It's fast to write, it's easy to reason about, and most of the time it will happen to line up with the thing you actually care about. The gap only shows up in exactly the case that matters most: someone removes the real thing but leaves an explanation of it nearby, which is one of the most likely ways a setting like this actually goes missing in practice. A comment explaining a rule is not the same claim as the rule being followed.

## What changed

The check got rewritten to look at the actual, structural presence of the setting rather than just scanning nearby text for its name. That's a more careful thing to build, and it was worth it, because the whole reason the check existed was to catch exactly the failure a text search could not.

Just as important as the rewrite was proving it. I didn't trust the new version just because it looked more careful. I deliberately broke the real setting again, with the same explanatory comment still sitting right there, and confirmed the check actually failed this time. Only after watching it fail on the real regression did I trust that it would catch the next one.

## What a person still has to decide

Writing the check itself can be done quickly, and a lot of it can be delegated. Deciding whether a check has actually been proven cannot. That takes a person choosing to break the exact thing the check is supposed to protect, on purpose, and watching what happens. A check nobody has ever tried to fool is a check nobody has actually tested.

## The rule worth keeping

Before you trust any safety check, break the thing it's supposed to catch, on purpose, with everything else left exactly as it was, including any comment that explains the rule, and confirm the check actually fails. A check that has never failed on the real problem it exists to catch hasn't been proven. It's just been written and hoped about.