Your First Live Migration Is Your Best Stress Test

The first reply to a 375-person mailer arrived about three minutes after the send. It came from a 63-team league asking us to run the conversion tool, and that tool had only ever touched a fake account.

What the tool does

Legacy volleyball sites on our platform were ordinary general-purpose sites tagged with a Volleyball subsport. We wanted them moved to a real volleyball sport type, with default stats, play-by-play scoring and a set-based scoreboard. A teammate built a staff page for it. It moves an account and its whole team tree across 37 per-sport tables with identity PKs preserved, because game, player and album IDs cross-reference each other. It also retags the sport columns on the customer and org-structure tables, moves photo folders on disk, and flushes cache rows.

The copy runs as one transaction: copy, verify counts, delete the source. Any mismatch rolls it back. It is dry-run by default, &commit=1 applies, and &photos=1 moves images. Hard stops cover ID collisions, and &force=1 overrides the data-loss warnings.

Scoping against production had shown 1,794 volleyball accounts, 753 of them active and 933 of them leagues. 36 of the 37 tables matched column for column. Only the stats tables differed, and only 3 accounts had any stats (32 rows).

The tool’s one test run was on a synthetic account. It migrated over to volleyball, rendered correctly (Set 1-5 scoreboard, a Sets column, a Play-by-Play tab), then migrated back byte-identical, every ID included. We closed the ticket.

The wrong turn: I sent the mailer first

With the tool “verified,” I moved on to the announcement. The mailer offered current customers an opt-in conversion and listed the new stats. I proofread the copy, fixed a button that opened a generic contact form instead of the conversion page, built screenshots from a purpose-built demo league, and sent it to 375 people. It promised the play-by-play stats to everyone regardless of package tier.

I hadn’t run the tool on a real customer, and I hadn’t checked whether play-by-play was gated by package. Both checks were still open, and I had already told 375 people what to expect.

The cost showed up within minutes. The first responder was a 63-team league, and we now had to run the tool on a live account and confirm the mailer’s claims against it, with 374 other customers reading the same promise. The mailer’s stats claim was already wrong on the day it went out, and a follow-up ticket to correct it came out of that.

Bug one: the gate that wasn’t there

The synthetic account had no package tier to speak of, so the test could never have shown this. On the live league, the Play-by-Play tab and the entry screen were both reachable on a package that doesn’t include them. Basketball and soccer already required a paid subscription for play-by-play. The new volleyball path never got the check, so stats visibility bypassed the tier restriction entirely.

The fix made volleyball play-by-play, both tab and entry screen, require the Prostyle subscription, the same as the other sports. The reply to the league had to say that the stats they were excited about need a package upgrade, which is not what the mailer told them.

A mock account would not have caught this, because the failure only exists where a real package and a real sport type meet.

Bug two: photos left behind

Every image URL on these sites is built from the sport name, in a path shaped like /photos/{sport}/{username}/. The database move retags the sport, so after &commit=1 every logo, banner and gallery URL points at a folder that doesn’t exist yet. Moving the folders was behind a separate &photos=1 flag, so a commit without it succeeded cleanly, verified its row counts, and stranded the images. Nothing errored, because the row-count check only covers rows.

The round trip on the fake account never exposed this. Migrating out and back with the same flags leaves everything consistent, including a photo folder that never moved. On a real account, the moment the data flips and the files don’t, the site quietly loses its pictures.

The fix inverted the default. Photos now move automatically on commit, and skipping that has to be requested explicitly and can’t be undone afterward. A destructive step should be the one you have to ask for.

A third trap: master versus league

Resolving the team tree from masterNode alone is wrong for a league that sits under a multi-sport org. There, masterNode points at the org. One such account returns 0 teams by master and 160 by league, and those teams would have been stranded in the old tables. The tool now resolves the tree as the parent chain UNION league UNION master, filtered to the sport.

That was already handled before the live run. It belongs here because it is the same class of problem as the other two: a rule that holds for the account you tested and quietly fails on the shape you didn’t.

What I do now

One live conversion took about an hour and twenty minutes, including verification, and it found two bugs that a passing round trip had hidden. I now treat the first real customer as the integration test and schedule it before the announcement, not after. Before that run I would have called the synthetic test thorough. It was byte-identical and it was wrong about the two things that mattered.

The lapsed-customer send, 1,086 people, was held back and split into its own ticket for review. It goes out after the conversion has survived a real customer, and the copy will say what the tier actually includes. A mailer is a promise made in bulk, and it can’t be recalled the way a bad commit can be rolled back.

dxdev LESSON · paste into your AI coding agent
LESSON: Run the First Real Customer Conversion Before Making a Bulk Promise
SOURCE: dxdev.com/blog/2026-09-26_migrations-as-integration-tests
WHAT HAPPENED: The tool moved a volleyball account and its team tree across 37 per-sport tables, preserved IDs, retagged sport fields, and could move photo folders. Its synthetic round trip passed byte-identically, but the mailer went out before a real customer migration and before package access was checked. On the first live league, play-by-play was available on a package that should require Prostyle, contradicting the mailer. A committed conversion could also leave photos inaccessible because database paths changed while folder movement required a separate flag. The tool was changed so photos move by default, and the lapsed-customer send to 1,086 people was held for review.
THE RULE: Treat the first real customer run as an integration test and complete it before announcing a migration or its benefits in bulk. Test the production combinations that connect data, entitlements, files, and organization structure, not only a reversible synthetic round trip.
CHECK MY CODE, then report PASS or FAIL with file:line for each:
      1. At least one representative live account has completed the migration and post-migration verification before any customer announcement is sent.
      2. The migration verifies package-gated features on the target sport type for the account's actual subscription tier.
      3. A committed migration moves required file assets by default and verifies that the resulting asset URLs resolve.
THEN PRINT: a table (check, PASS/FAIL, evidence, fix) + a verdict (applies / partially / OUT_OF_SCOPE / no) + the single most important next action.