---
title: "What the First Real Account Found That Sample Data Missed"
canonical: https://dxdev.com/blog/first-real-run-found-the-gap/
datePublished: 2026-08-19
---
The first real customer account we ran through the conversion tool turned up two problems that testing had never touched. The tool had already been built and tested. It moves an account from a generic configuration onto its category's own dedicated code, the same move every account in that category will eventually need. Sample data had never exercised either problem.

## The gate that was never built

This category has a stats feature, entry of detailed data. Two comparable categories both restrict it to a paid tier. This one didn't restrict it at all. Every account on it, whatever the tier, had full access to a feature that was supposed to require an upgrade.

The account we were converting was on the entry tier, so the gap showed immediately. After the fix, that account lost access to a feature it had never been entitled to and never should have had.

Nothing we found recorded this as an intentional exception. It looked like a gap in the category's original build that nobody had closed, in code the other two categories got right from the start. It surfaced now because this was the first time a real entry-tier account ran the detailed stats feature through this code path with anyone checking whether it was gated.

## The second gap: photos left behind

The conversion also has to move photo assets to the new storage location. That step was a separate option, and the main move didn't enforce it. If you skipped it, the photos stayed behind and nothing complained. There was no failed job and no flag, and nothing a future run or a support ticket could use to show the conversion was incomplete.

It only bites when someone runs the conversion without remembering the extra step. Sample data never makes anyone remember it.

## What verification looked like

We checked both fixes against the real account on both sides of the change.

- **Tier gate, before:** the gate was open and the feature worked without a paid tier.
- **Tier gate, after:** the gate applied, and a comparable paid-tier account kept full access exactly as before.
- **Photos:** a [live dry-run against the real account](/blog/dry-run-that-lied-different-code-path-from-apply/) confirmed the tool now moves photos by default. That day's real conversion had already exercised the underlying move code the fix now defaults onto.

## The gate fix collided with a mailer

Earlier, we had sent a conversion mailer to a large batch of customers. It told everyone the detailed stats feature came with their account regardless of tier. That was true, but only because the gate had never existed. Once the tier gate was fixed, the promise was false for anyone below the paid tier.

## Why the first real run needs its own review

A suite built on synthetic accounts would never have caught either problem. Synthetic accounts don't carry the accumulated shape of years of real usage. Here that meant a tier nothing had ever had reason to check, and an optional step nothing had ever forgotten to take.

A clean test environment can't expose assumptions that were only true because nothing real had tested them yet. The first real account does.

## What to check

Treat the first run of any migration against a real customer account as its own review. Check which paid-tier entitlements are actually enforced, and which optional steps can be skipped without an error. A missing tier gate made an earlier mailer's promise false, so check what customers have already been told before you close a gap like this one.