--- title: "What an August 7 Cutover Audit Turned Up in Our Cloudflare Registrations" canonical: https://dxdev.com/blog/cloud-api-silent-deletion-guard/ datePublished: 2026-07-20 --- At 8:36 on July 20, we went in to answer one question: would five client-managed domains break at an August 7 cutover? Two would. Three would not. Before we closed it, we found **55 Cloudflare custom-hostname registrations that no longer existed**. They had not failed visibly. They were gone. A failed registration gives you something to inspect: an error state, a retry, an API response, a bad certificate. A silently deleted one gives you an empty space. Our dashboard could show the registrations it still knew about. It could not show the ones Cloudflare had removed when an endpoint changed. Months can pass in that gap, right up until a customer hits the broken path before we do. We fixed the 55. The longer job was making the same kind of disappearance visible every day. ## The inventory was lying by omission Five domains had been partially migrated. We traced the origin path for each one and worked out which setup would be live on August 7. That gave us the immediate answer: two at risk, three safe. The trace also forced a comparison we had never been making continuously. We have a product-side set of domains that should have a Cloudflare custom hostname. Cloudflare has its own set of registrations that actually exist. Neither of those is the status field inside our application. For this configuration, the expected Cloudflare object is `www.`. The apex is not a Cloudflare custom hostname. That detail is small enough to get lost in a migration script and large enough to turn a correct audit into a stream of false positives. The invariant is just as small: ```text expected Cloudflare hostnames - actual Cloudflare hostnames = registrations to investigate ``` The result was 55. Nothing in the dashboard was wrong in the ordinary sense. It was reading the registrations that existed. The mistake was treating that view as an inventory. A dashboard is a projection of the records that survived, and it can't tell you the remote system still holds every record you expect. There is no row labeled "this used to exist." The absence has to be computed against an independent expected set. ## Repairing the write path The immediate repair was a bulk re-registration of the 55 missing entries, verified against the live system. That restored the objects, but a bulk repair alone would only have set up the next incident. The durable fix went into `CFWrite`, the path that manages the registration. It now handles a deleted registration as a **purge and recreate** operation. When Cloudflare has removed the remote registration, we discard the stale registration state and build the remote object again. Keeping the old status and updating in place would have left a healthy-looking record for the dashboard and no object behind it. The staff work mattered too. The dialog and help content now distinguish customer-fronted domains. That classification has to come before the comparison. If a customer owns the front of a domain, it must not be added blindly to the expected Cloudflare set. Otherwise the guard would be technically active and useless in practice, because it would teach staff to ignore its false alerts. ## The guard that landed on a workstation The repair path handles a known missing registration. It doesn't prove that tomorrow's endpoint change leaves the correct remote state behind. For that we added `CF-RegistrationGuard`, a daily sweep that runs the expected-versus-actual comparison. The first scheduled task landed on a workstation. It ran, which proved the task could execute, and that was all it proved. A workstation sleeps, reboots, loses a session, and belongs to a person. We removed that version and put the daily guard on the origin box, where the rest of the domain automation lives, then verified the corrected deployment by count. Where the check runs is part of the design. A check only counts as continuous when its host, schedule and execution context belong to the system it is checking. A daily sweep also covers what an event-only check would miss. We could validate only when `CFWrite` changes an endpoint. That can't detect drift that happened before the new code shipped, and it asks the same remote call to both change state and judge whether the provider kept it. The sweep asks independently what we expect today and what Cloudflare actually has today. ## Checking the checker We did not leave the guard as an untracked script on the server. `CF-RegistrationGuard` and its companion detector are in the engine file list and the committed deployment manifest. A later parity check confirmed the engine files on the origin box and in the repository were byte-identical. That matters because a daily verification script is only useful if production runs the version you think it does. Otherwise you can build a good guard, commit it, and still operate a server with yesterday's assumptions. If you depend on a provider that can delete state quietly, ask whether anything compares what you expect to exist against what the provider holds, on a schedule, from production infrastructure. Then confirm the deployed copy of that check matches the committed one.