Two Weeks of Error Pages Behind One Customer's Short Address

For two weeks, anyone who typed one customer’s short address got an error page instead of their homepage. The customer never saw it, and nobody reported it, because the other ways into their site worked fine the whole time.

We found it by trying to break our own work. Nobody had complained.

The rule that worked

A few weeks earlier we had been closing out a run of scanner traffic hammering the platform’s edge. Vulnerability scanners probe the same short list of paths on every site they touch: WordPress admin routes, a handful of framework debug endpoints, and a scattering of generic strings that only ever show up in recon tooling. The fix was a blocklist rule that matched those exact strings and dropped the request before it reached the origin server. It worked. Failed requests reaching the origin dropped hard, and we moved on to the next scanner in the queue.

One of the strings we added was a short, unremarkable path segment. Across a week of logs, on every site the platform hosts, it had shown up zero times as a legitimate 200 response. That was the bar we had set for adding anything to the rule, and zero looked like proof it was safe.

It was proof of something else. The platform gives every customer a canonical short URL built straight from their own username: one path segment, nothing else. The string we had blocked as a probe signature was also, by coincidence, one customer’s actual username. Their front door, the address printed on a flyer or pasted into an email, matched the rule and got dropped at the edge.

Why it stayed hidden

Other routes to that customer’s site kept working. Only the short address failed, and every uptime signal we had only ever exercised the working routes. The account looked healthy from every angle we were checking.

The pass built to disprove it

What surfaced it was a habit rather than a monitor. After shipping a rule like this, we run a pass designed to disprove it instead of confirm it. We went looking for the failure we would have argued against if asked directly: could any string on that blocklist also be a real username? We checked the full list against the actual customer table instead of trusting that “obviously a scanner path” meant anything. It took one query to find the collision.

We removed the clause instead of narrowing it. A scanner probe path is a nuisance. A paying customer’s homepage returning an error is a failure we will not accept as a tradeoff, so the obscure signature lost and the customer’s address won. We then checked every other value still on that rule against the customer table, close to thirty entries, and found no other collisions. That told us this was a one-off coincidence and not a pattern to redesign around.

The summary that skipped clauses

There was a second bug in there that we did not expect. The tool we had been using to summarize what was on that rule understood only one kind of match clause and skipped the others without any warning. The count it reported had been wrong for a while, and it made the rule look shorter and easier to review than it was. That is part of why the collision sat unnoticed: the tooling we trusted to tell us what the rule contained was not telling us the whole rule. We fixed it in the same sitting, since a summary tool that drops entries silently is worse than having none.

What the check actually answered

An exact-match rule on a short string is only as safe as the space of real values it might also equal. “We checked it wasn’t getting legitimate 200s” answers a different question than “we checked it wasn’t someone’s actual address.” The two look like the same check. The gap between them is where a real customer sat, locked out of their own short address, for two weeks.

So before a block rule goes live, we now check every string on it against the customer table as well as against past traffic. We also confirm that the summary we review shows every clause on the rule, and we run a pass built to disprove the rule after it ships. A customer can be locked out of their own address while every uptime signal stays green.

dxdev LESSON · paste into your AI coding agent
LESSON: An Exact-Match Block Path Can Collide With a Real Address
SOURCE: dxdev.com/blog/block-rule-also-blocked-a-paying-customer
WHAT HAPPENED: An edge firewall rule blocked a fixed list of short paths known to be scanner recon noise. One of those short strings was added in good faith as a probe signature, then turned out to also be a real customer's own short URL on a platform where every account gets a route named after its username. One variant of that address resolved differently from an equivalent one, and every uptime check only ever exercised the working variant, so the account never looked down from the outside and no monitor caught it. It stood for about two weeks until an adversarial verification pass, run specifically to try to break the assumptions behind a recent rule change rather than confirm them, checked the rule's values against the real list of customer usernames and found the collision. The rule was removed rather than narrowed, every other value in the same rule was swept against the same list with no further hits, and a related tool that had been undercounting the rule's own clauses (which helped the collision go unnoticed) was fixed in the same pass.
THE RULE: Before adding an exact-match or suffix-match path clause to a rule that blocks by string rather than by behavior, check that string against every real identifier your own system generates routes from (usernames, slugs, account codes). One representation of an address resolving differently from an equivalent one is exactly the kind of collision that won't show up as an outage, because monitoring built around a single representative URL only ever exercises the form that already works.
CHECK MY CODE, then report PASS or FAIL with file:line for each:
  1. Any firewall, WAF, or app-level blocklist rule using exact-match or suffix-match on a short path segment, with no check against the live table of user-generated route identifiers (usernames, slugs, org codes).
  2. Any monitoring or uptime check for a route family that treats one URL variant as representative of the whole family, when different variants could resolve differently.
  3. Any tool that counts or reports the contents of a rule, ruleset, or config list by parsing only one clause type (e.g. only "contains" matches), when the underlying rule can hold other clause types it silently skips.
THEN PRINT: a table (check, PASS/FAIL, evidence, fix) + a verdict (applies / partially / OUT_OF_SCOPE / no) + the single most important next action.