A 530 from the archive MCP that pointed at the wrong component

Jul 15, 2026 · 3 min read

Updated Oct 13, 2026

Cover image for: A 530 from the archive MCP that pointed at the wrong component

At 10:59 PM, our archive MCP was returning Cloudflare 530 instead of serving requests.

The manual recovery took two minutes. The failure had started much earlier, at boot, when the tunnel tried to come up before the machine had a usable internet connection. It exited. Nothing started it again. By the time an agent needed the archive, the public edge was alive enough to return an error, but the route behind it was gone.

We restarted the tunnel and the MCP came back immediately. Then we opened a high priority follow up to make the startup path recover from an internetless boot. The restart restored service, but the condition that caused the outage was still there.

What the 530 pointed at

The visible error made this look like a Cloudflare problem. It wasn’t one.

The archive MCP depends on a tunnel to make the service reachable outside the host. That puts cloudflared directly on the request path:

agent request → Cloudflare edge → cloudflared tunnel → archive MCP

A failure at any point can look the same to the caller. A bad MCP endpoint, an authentication problem, an unavailable upstream service and a dead tunnel can all end with an agent unable to reach the archive. The 530 did not name the broken component. It only marked where the request stopped being useful.

The detail that mattered was the boot sequence. cloudflared fired while the internet was not ready. It exited, and no later event retried it. A short network timing gap became a persistent outage.

So the tunnel failed once, early, and stayed absent until we stepped in. That changes the repair. Watching for a live process after the system settles would not explain why it never came back. Restarting the process at 10:57 PM did not explain why it had failed in the first place.

Two events boot treated as one

Boot treated these as interchangeable:

The operating system had started the tunnel process.
The host had a usable route to the internet.

They are different events.

A restart policy answers “What happens after a process fails?” A startup dependency answers “What must be true before this process makes its first attempt?” Infrastructure often gets the first and is called resilient. Here the first attempt was allowed to happen in the wrong state. The process exited and the service did not recover on its own. The fault sat in the startup chain, not in the code that serves the archive.

The fallback that already lost

The old design had a real fallback: notice the 530, restart the tunnel, move on. We had been relying on it, and it worked in two minutes. It also needed a human or an agent to be looking at the right failure at the right time.

That is where it lost. Manual restart turns a transient boot condition into an operational dependency. The machine can ride out an ordinary connectivity delay, but the service can’t, because nothing in the service owns the recovery path.

A generic restart-on-failure rule alone would also fall short. It may cover a process that crashes after a healthy connection, but it says nothing about the first connection attempt waiting for the right precondition. A boot chain needs both parts: an ordering rule that respects network availability, and a retry or restart path that survives the network changing after the process begins.

We are not pasting service configuration here. The incident record establishes the failure mode, the two minute manual recovery and the follow up. It does not establish an exact service manager configuration or a finished deployment, and inventing one would turn an incident note into made-up architecture.

What the follow up has to prove

The permanent repair is to make the tunnel’s lifecycle match the dependency it actually has.

The startup chain must not treat an early missing network as a terminal failure. If the initial connection can’t be established, the infrastructure has to make another attempt without waiting for someone to notice an outage. If the network arrives after the first attempt, the tunnel must still become available.

That requirement is deliberately narrower than rebuilding the MCP stack. The archive service recovered the moment the tunnel was restarted, so the evidence points at the transport layer and its boot timing. Changing application code, changing the archive, or treating the edge error as a Cloudflare configuration problem would widen the blast radius without touching the sequence we observed.

The follow up exists because the first recovery was manual. The next one has to happen before an agent gets a 530. The success test is simple: boot with no internet, let the network arrive, and verify that the archive MCP becomes reachable without anyone restarting cloudflared.

A restartable service is not a self healing one

A service is self healing when the dependency it needs can arrive late, fail early, and still leave the service available when the work starts. Boot a machine with no internet and check that the service becomes reachable without a manual restart. If it doesn’t, you have a process that can be restarted, and that is a weaker thing.

AI Skills

Use this lesson with the AI assistant you already use

A tunnel process fired at boot before the machine had a usable internet connection, exited on that failed first attempt, and nothing ever retried it, so a two-minute network timing gap silently became a persistent outage until someone noticed hours later.

Paste the prompt, share only the context needed to answer it, and treat the result as a draft for your review. Do not include confidential information or let an AI assistant make changes without your approval.

Optional: for a visual report and saved memory, run /dxdev first. Don’t have it? Get it at dxdev.com/skills/dxdev. The prompt works without it.

dxdev LESSON · paste into your AI coding agent
LESSON: A restart policy answers what happens after a process fails, not what must be true before its first attempt
SOURCE: dxdev.com/blog/vault-mcp-boot-timing-failure
WHAT HAPPENED: A tunnel process making an internal service reachable through a public edge fired at boot before the machine had a usable internet connection, exited immediately on that failed first attempt, and nothing was ever configured to retry it, so the service silently stayed unreachable from that point forward. The failure surfaced hours later as an edge-level error code that looked like a third-party provider issue but was actually a dead process nobody had restarted. Manually restarting it fixed the immediate outage in two minutes, but that fix was treated as insufficient on its own, since it converts a transient boot-timing condition into an ongoing operational dependency on a human noticing at the right moment. The real fix required separating two guarantees that had been conflated: the operating system having started a process is not the same event as the host having a usable route to the internet, and a startup chain needs both an ordering rule that avoids attempting the connection before the network is actually ready, and a retry path that survives the case where network availability changes only after the process has already made its first attempt and lost that race.
THE RULE: A process failing once at boot, before its actual dependency was ready, and never being retried is a different failure mode from a process crashing after it was already healthy; a restart-on-crash policy alone does not cover a process whose very first attempt happened in the wrong state and simply gave up, so infrastructure needs an explicit precondition check before that first attempt, not only resilience after it.
CHECK MY CODE, then report PASS or FAIL with file:line for each:
      1. Any service or process started at boot with no explicit check that its actual dependency, such as network connectivity or a downstream service, is ready before its first connection attempt.
      2. Any process supervisor or restart policy that only reacts to a process crashing after it was previously healthy, with no coverage for a process that exits cleanly on a failed first attempt and is never retried.
      3. Any outage investigation that stops at a generic edge-level or provider-level error code without tracing the request path back to the specific component, and specifically its boot sequence, that actually failed.
THEN PRINT: a table (check, PASS/FAIL, evidence, fix) + a verdict (applies / partially / OUT_OF_SCOPE / no) + the single most important next action.

Send us what your agent found →

Prefer to talk it through first? Start an assessment or talk to us.

Lesson: A restart policy answers what happens after a process fails, not what must be true before its first attempt

Choose your next level of detail

Want the practical version?

How AI Helps starts with the work problem, what improves, and where human judgment still matters.

See how AI helps Talk to us

A 530 from the archive MCP that pointed at the wrong component

What the 530 pointed at

Two events boot treated as one

The fallback that already lost

What the follow up has to prove

A restartable service is not a self healing one

Use this lesson with the AI assistant you already use

Related in Silent failures

Why 1,282 Custom Hostnames Did Not All Get Retried

The one-time Chrome launch that stranded session 11 without a browser

Cloudflare error 1001 on a site whose CNAME was correct

The Test That Passed 497 Seconds of Runs Without Ever Touching the Guard

check:rls exits 1 after every listed check passes

Want the practical version?

Follow the Build