Two alerts, 25 domains, and a monitor watching the wrong sites

At 7:10 a.m. the apex relay health monitor fired. It fired again at 7:30. By 3:38 p.m. we had moved 25 brand domains, including the primary domain, off the old origin server and onto the apex relay and Cloudflare. The sites stayed up. Mail stayed intact.

This was not a routine DNS change. We run a partnership business with staff and customer facing systems, and we have no dedicated operations team to absorb an infrastructure migration. The same people who own the product and the customers have to make the infrastructure change legible enough to operate.

The hard part was deciding which system would notice each change first, and making every transition reversible before the next one began. For over six hours the primary domain and the mail for a live business were exposed to every step. Adding 25 domains to Cloudflare was the easy part.

The alerts marked a boundary

The morning began with timeout alerts from the apex relay. At the same time a customer reported that a domain was not working, and the www CNAME was not pointing where it should. Those facts looked connected, but they were not evidence of the same failure.

A relay health alert says a monitor cannot complete its check. A bad www CNAME says one hostname is not following the intended routing path. Neither tells you, by itself, whether the problem is the old origin, Cloudflare, the relay, the DNS record, or the monitor’s own target.

That changed the work. We did not start by silencing the alerts or by making a broad nameserver cutover. We mapped everything that could report success or failure: the apex hostname, www, the relay, Cloudflare, mail, and the health monitor. For every domain we needed to know what received traffic, what resolved it, and what observed it.

The output was an inventory of all 25 brand domains and a migration recipe committed to the repository. The inventory was the control surface for the move. Without it, a DNS record that looked safely changed could still leave a www alias behind, or a monitor could keep testing a path that no longer represented production.

We refused the one shot cutover

The tempting route was a bulk migration: make all 25 DNS changes, turn on the new Cloudflare configuration, then clean up whatever alerts appeared. That is fast only if nothing is wrong. Once something is wrong, the batch has erased the evidence. You no longer know which domain introduced the failure, which endpoint is still serving the old origin, or whether a rollback is restoring the right state.

The existing health monitor was a second problem. It was checking personal and expired test domains, and it had been checking those instead of the company domains customers actually use. We have no number for how long, and it had nothing to do with this migration. That is a monitoring gap with consequences. A real production outage on any of the 25 company domains, at any point before this migration, could have gone undetected because the thing meant to watch for it was watching something else. A green result there would not have shown that the company domains were healthy. A red result could have sent us chasing an obsolete target while the production relay was fine.

Mail was the other temptation. The immediate work was web routing, so it would have been easy to call mail out of scope. That would have been another bad boundary. The requirement was that pages loaded through the new path and that moving DNS and traffic did not disturb mail.

So the sequence became the architecture. First we established the inventory and the known routing path. Then we moved each domain from the old origin to the apex relay and Cloudflare. At each checkpoint the next system had to be able to notice the change: DNS had to resolve to the intended path, the relay had to receive traffic, Cloudflare had to sit in the new route, and mail had to remain intact. Only after the production path was established did we point the health monitor at the company domains.

That last step matters more than it sounds. Monitoring is part of the routing design. If it checks the wrong hostname, it tells you nothing useful. If it changes before the route is verified, it adds noise while you are trying to understand the deployment.

The rollback was the next checkpoint

Two alerts occurred before the migration was complete. The useful definition of zero downtime here was that every change had a known prior state and a bounded next state.

For a domain, rollback meant the old origin path still existed until the relay and Cloudflare path had been verified. For monitoring, it meant we did not swap a stale signal for a different opaque one. We moved the service path first, then pointed the monitor at the domains that represent the service customers use.

This is why the www CNAME mattered. A domain is a set of hostnames, records, targets and observers. If the apex has moved but www still points elsewhere, the migration is half done. If the health check is still on an expired test domain, the operational view is half done too.

By the end of the 6 hour 24 minute work session, all 25 domains were on the apex relay and Cloudflare, the primary domain had moved without downtime, mail was intact, and the health monitor had moved off the personal and expired test domains onto company domains. We closed the migration item with the full inventory and recipe in the repo so this does not become tribal knowledge.

The complexity was already there, in the relationship between DNS, routing, mail and monitoring. Writing down the sequence made that relationship inspectable. The next migration starts with a record of which component needs to notice each change, what counts as verification, and what can be restored before the next change proceeds.

What the monitor was watching

Before any infrastructure move, check what each health monitor is actually watching. Ours was watching personal and expired test domains instead of the 25 company domains customers use. Get an inventory of every hostname, including www and mail, and a known prior state for each change, so a failure traces to one step and can be undone before the next begins.

dxdev LESSON · paste into your AI coding agent
LESSON: A Health Monitor Watching the Wrong Domain Is Worse Than No Monitor
SOURCE: dxdev.com/blog/zero-downtime-domain-orchestration
WHAT HAPPENED: A live migration of 25 brand domains (including the primary domain) from an old origin server to an apex relay plus Cloudflare was done without a dedicated ops team, using an inventory of every domain's apex, www CNAME, relay, Cloudflare, and mail dependencies as the actual control surface for the move. The existing health monitor was found to be checking personal and expired test domains, not the company domains customers use, a pre-existing blind spot that could have masked a real production outage at any point before this migration. The fix sequenced the monitor's cutover to company domains only after the new routing path was verified, so a false-positive-looking green result on the old check couldn't be mistaken for real coverage during the move.
THE RULE: A health monitor's coverage has to be verified against what it's actually checking, not assumed from its existence; a monitor pointed at the wrong target (an expired test domain instead of the production one) is more dangerous than having no monitor, because it manufactures false confidence that something is being watched, and that gap can persist indefinitely with nobody noticing until an unrelated migration forces an inventory of what's actually being checked.
CHECK MY CODE, then report PASS or FAIL with file:line for each:
  1. Any health check, monitor, or alert whose target has not been re-verified against actual current production domains/endpoints in the last major infrastructure change.
  2. Any infrastructure migration plan (DNS, routing, origin) that does not include an explicit inventory of every dependent hostname (apex, www, mail, subdomains) before changes begin.
  3. Any migration sequence where monitoring is cut over to the new state at the same time as the underlying infrastructure, rather than only after the new path is independently verified.
THEN PRINT: a table (check, PASS/FAIL, evidence, fix) + a verdict (applies / partially / OUT_OF_SCOPE / no) + the single most important next action.