Every LLM-backed feature you ship should have an off switch the day it goes live, not the day it catches fire in production. In my own work, a commitment-inference loop checks one environment variable before it does anything else, and that single flag does three jobs that would otherwise be three separate projects.

The whole pattern is the first guard at the top of the inference function:

if (process.env.MOCK_INFERENCE === "1") {
  console.log("[inferenceHandler] MOCK_INFERENCE=1, returning mock");
  return MOCK_RESULT;
}

That is it. Flip the flag and the function returns canned JSON before the LLM call. No OpenAI call, no network round trip, no API key required, no downtime. The whole feature, an LLM that reads a user’s posted update, infers how a node’s state should change, and persists the result, can be turned into a deterministic stub by setting one variable.

People treat the kill switch as cleanup, the thing you bolt on after the postmortem, and they have it backwards. The kill switch is the cheapest piece of infrastructure in the entire feature and it is most useful before you have an incident, not after, so build it first.

Three jobs, one flag

MOCK_INFERENCE=1 earns its place because it is one branch in the code that solves three different problems at the same time.

Job one: instant production rollback. When a model starts misbehaving in prod, returning garbage, hallucinating a state transition, latency-spiking on the OpenAI side, you do not want your remediation plan to start with “open a PR.” You want to flip a variable and have the inference path go quiet immediately while you keep the rest of the app running. The feature degrades to canned output instead of taking the app down with it. No redeploy, no downtime.

Job two: zero-cost local dev. A new contributor, or me on a plane, or a CI runner, should be able to boot the app and exercise the whole flow without an API key and without spending a cent on tokens. With MOCK_INFERENCE=1 set in the local env, the inference loop returns the canned mock response and the rest of the system, the API procedure, the DB write, the node patch, the UI, all runs exactly as it does in prod. You develop against the real shape of the data, not a fake.

Job three: deterministic tests. Non-deterministic GPT calls are the enemy of a green test suite. With the boundary stubbable, the vitest suite mocks the LLM caller and feeds it both valid and invalid payloads, then asserts the validation behaves. You are not testing the model, you cannot. You are testing every line of your own code that surrounds it, which is exactly the code that breaks.

One env var, three problems that would otherwise be three projects. That return only exists because the switch was designed in alongside the feature, not retrofitted after the incident it would have prevented.

The boundary is where the discipline lives

The kill switch is only half the story. The other half is what happens when the LLM does run and returns something real. The output of a language model is untrusted input. It is a string that claims to be JSON that claims to match your schema. Treat it the way you would treat a request body from a stranger.

So the loop is: infer, persist the raw update, then patch the node only if the inference validated. The LLM returns JSON, a zod schema validates it, and the inference function returns null the moment the parse throws. The full persistence procedure always saves the raw-text row first so the user’s words are never lost, applies the inferred patch to the node only when validation passed, and throws a 422 to the caller when it didn’t. The model’s guess never silently mutates a node, because the garbage stops at the schema before it ever reaches the database.

This is the part people skip, and it is the part that bites. Without the zod gate, a malformed model response either throws somewhere deep in your persistence layer or, worse, quietly patches a node with a half-broken state you discover three weeks later. With the gate, an invalid inference is a clean 422 at the boundary, the raw text is still saved, and the node is left untouched. The validation schema and the mock are the same idea applied at two ends of the same pipe: one defines the shape going out under test, the other enforces the shape coming back in production.

The rollback ladder

The honest version of “we can turn it off” is more than one lever, and I wrote the whole ladder into the commit message so the next person, probably future me at 2am, does not have to reconstruct it. Three escape hatches, escalating in cost:

  1. Flip the flag. MOCK_INFERENCE=1. No downtime, no deploy, instant. The feature returns canned output and the app stays up.
  2. Drop the schema. Roll back the migrations that added the inference tables and columns. This tears out the data the feature added when the problem is in the model of the world, not the inference call.
  3. Revert the code. git revert. The full undo when the feature itself was a mistake.

You reach for them in that order because they cost more as you go down. Most incidents never get past rung one. The point of writing the ladder down at ship time is that you decide the rollback story while you still understand the feature, not while it is on fire and you are guessing which migration added which column.