---
title: "A dry run is only useful when it checks the same things as apply"
canonical: https://dxdev.com/blog/dry-run-that-lied-different-code-path-from-apply/
datePublished: 2026-05-26
---
A dry run should answer a straightforward question: **what would happen if I ran this change now?**

That question becomes dangerous when the preview takes a shortcut around a precondition that the real operation checks. The dry run may report success because it never checked whether success was possible. The real run then fails-not because the environment changed, but because the two modes were answering different questions.

A reassuring preview can be worse than no preview. With no preview, people know they are operating with uncertainty. With a misleading one, they proceed with confidence they have not earned.

## The failure mode

The pattern is easy to create. A function branches on a `dry_run` flag near the beginning:

```python
def prepare_change(target, dry_run):
    if dry_run:
        return "would succeed"
    if not target.is_ready():
        return "cannot proceed"
    return perform_change(target)
```

The apply path checks whether the target is ready. The preview does not. Both paths may claim to answer “can this change succeed?”, but they use different predicates. That is not a simulation; it is a second program with a friendlier message.

The safer shape is to evaluate all preconditions before the only difference between modes:

```python
def prepare_change(target, dry_run):
    if not target.is_ready():
        return "cannot proceed"
    if dry_run:
        return "would succeed"
    return perform_change(target)
```

The examples are deliberately simple. In real systems, readiness may include authorization, input validation, ownership, availability, concurrency or lock state, policy checks, dependencies, capacity, or the ability to recover. A dry run should evaluate the same relevant conditions the real change will evaluate. Only the side effect should be omitted.

## Treat a mismatch as a stop signal

A preview and an apply run will not always produce identical results. The environment can change between them, external services can be unavailable, and a concurrent user can alter the state. Those are real limits that a tool should disclose.

But a systematic mismatch is evidence. It should stop the operation until someone can explain it. Do not select the explanation that lets the change proceed; investigate the difference between predicted and observed outcomes.

A good workflow is:

1. Run the preview against a controlled, authorized scope.
2. Record the expected actions, expected skips, and reasons for every decision.
3. Validate the same logic with representative, non-sensitive test data.
4. Require a responsible person to review the plan before a consequential operation.
5. Apply only after the review, using the same scope and guardrails.
6. Verify the actual outcome, preserve an audit record, and stop if the result diverges materially from the plan.

For destructive or high-impact operations, add proportionate safeguards: least-privilege access, explicit authorization, idempotency, transaction boundaries where appropriate, backups or recovery points, rate and resource limits, monitoring, alerting, rollback criteria, and a clear incident path.

## Design for explanation, not only counts

A preview that says “161 items would change” is less useful than one that can explain which items would change, which would be skipped, and why. Counts are summaries. The decision record is the evidence.

That record also makes it possible to distinguish a legitimate state change from a logic defect. If the preview says an item is eligible but the real operation rejects it for a precondition the preview never evaluated, the report reveals the gap immediately.

The same principle applies to automated systems and AI-assisted workflows. A suggestion, a plan, or a simulation is evidence for a person to review. It is not permission to bypass the ordinary controls around data, identity, authorization, and irreversible change.

## The takeaway

A dry run earns trust only when it is a faithful application run minus the side effect. Share the decision logic. Delay the `dry_run` branch until the final write, call, delete, or update. Make preconditions and expected skips visible. Then treat any meaningful difference between prediction and outcome as a reason to pause, investigate, and improve the tool before trying again.

## Related

- [Let AI suggest before you save](/blog/dry-run-llm-inference-before-you-persist/): preserving a human decision boundary before durable data changes
- [The runtime was half dead: migration as audit](/blog/the-runtime-was-half-dead-migration-as-audit/): why artifacts and observed effects are better evidence than a successful-looking status message