Why Our Hotfix Train Ran Without a Single Pull Request

Between 9:13 in the morning and 5:30 in the afternoon we cut six tagged production versions of a legacy application: a minor release and five hotfixes on top of it. No CI gate. No pull requests. No staging soak. Each one merged straight to master, tagged, and pushed to a build server that deployed it.

If you came up through a shop with branch protection rules and a green-checkmark culture, that reads like a confession. What made the day manageable was what each tag carried and how cleanly it could be taken back.

The cadence

The day started with a minor release cut around 9:37. That is the normal flow: develop gets a version wrap, it becomes master, it gets a plain version tag, and the build server picks it up.

Then the train. Over the next several hours there were five hotfix/ branches. Each was cut from master, carried exactly one ticket, was merged back to master and tagged, and was merged into develop within minutes of being cut. The version numbers climbed one patch digit at a time.

The map is the whole point:

• hotfix 1 carried one ticket
• hotfix 2 carried one ticket
• hotfix 3 carried one ticket
• hotfix 4 carried a tightly-coupled pair
• hotfix 5 carried one ticket

You can name the ticket behind each of the five tags without opening the diff. The whole train runs on that rule.

One ticket per tag is a rollback strategy

When you ship straight to production with no soak, the question is this: when something breaks at 4pm, how fast do you get back to known-good, and how much working code do you destroy on the way?

Batch five fixes into one release and the answer is bad. The live tag contains five changes and one of them broke. You can revert the whole tag, which kills four good fixes that customers are already using. Or you can dig through the diff to pull out the one bad change while the site is degraded and you are under pressure. Neither is fast and neither is clean.

One ticket per hotfix tag collapses that. If the third hotfix broke, it was one ticket. The rollback is “revert to the tag before it,” a single step that keeps every other fix from the morning. The rollback scope is bounded by that tag’s change set, because the version granularity stays close to one ticket.

That is the trade we keep coming back to: version granularity is your blast radius. Coarse versions mean coarse rollbacks. If your tags are as small as your tickets, your worst-case recovery is as small as your tickets too. On a legacy app with uneven test coverage, a small unit of rollback is especially valuable next to the local verification already in the release process.

Cut from master, then merge back into develop

The easy version of the branch choreography is also the wrong one: hotfix straight onto master, ship, forget. Do that five times and develop sits five fixes and five version bumps behind production. Your next minor release either silently loses those fixes or fights a merge conflict over version metadata.

What keeps the two branches honest is to cut from master, version-wrap, then merge the tag back into develop. For each hotfix:

1. Cut a hotfix/ branch from master.
2. Land the one ticket’s change.
3. Merge to master, bump the version, tag it.
4. Merge that same commit back into develop so develop carries both the fix and the version bump.

Skip step 4 and you have started a slow leak. Master and develop diverge on the thing that is hardest to reconcile later, the version number, and the next person to cut a release inherits a conflict they did not cause. The merge-back is cheap insurance that keeps “ship from master” from poisoning “build from develop.”

Three people on one train

There were three of us on the codebase at once. We interleaved without collisions because nobody worked on a shared branch. Every change lived on a dedicated feature/ or hotfix/ branch named for its ticket, and the branch name was the lock. Our change sat on our branch and someone else’s sat on theirs, so we could all be merging to master in the same hour. The only place we met was the merge itself, which git is good at flagging.

One ticket per branch also avoids the merge-hell that “everybody on develop” produces, and nobody has to announce what they are touching.

The author is the QA

So where did the staging soak go? It moved.

“No staging” is carrying a lot of weight in that opening paragraph, so here is what sits behind it. Before any of those hotfixes got cut, the fix ran on a local IIS clone of the app and we demoed it working with our own eyes. Click the thing, watch it do the right thing, then cut the tag. The verification step is real. The person writing the code watches it work on a clone that mirrors production, with no separate environment and no separate QA pass.

That is the load-bearing assumption, and it is exactly where this model stops scaling. It works because the author can fully exercise the change locally and trust what they see. Once a fix has effects you cannot observe on a clone, such as a background job, a payment webhook, or something that only misbehaves under real traffic, the author-as-QA model no longer covers you and you have earned a soak. For a UI tweak or a query fix you can drive by hand, the local demo is the soak. For anything you cannot see happen, it is not.

When the train stays in the station

The same day had a counter-example next to the hotfixes: a net-new analytics surface, several thousand lines across a clutch of new pages. It all landed in a single feature branch that merged to develop and rode the next minor, with no hotfix tag of its own. That was correct. Hotfixes are for changes with production-behavior risk that you want to ship and roll back atomically. New pages that nothing else depends on have no existing behavior to break, so there is nothing to roll back independently and no reason to spend a version number. They merge to develop and wait for the next minor.

The rule that falls out: hotfix-per-ticket is for changes that touch live behavior, and greenfield rides the minor. Knowing which bucket a change is in is most of knowing whether it gets its own tag.

What to ask before you ship straight to production

Ask how many changes each release carries, because that number sets how much you lose when you roll back. One ticket per tagged release keeps a rollback to one step. Then ask who verifies each change and where. Here it was the author on a local clone, which covers what can be driven by hand and leaves out what only shows up under real traffic.

dxdev LESSON - paste into your AI coding agent
LESSON: One ticket per hotfix tag is a rollback strategy, not bookkeeping
SOURCE: dxdev.com/blog/six-production-releases-before-lunch-hotfix-train
WHAT HAPPENED: A small team shipped five consecutive hotfixes across one day on a legacy app with no CI gate.
Each hotfix branch was cut from master, carried exactly one ticket, merged back to master, tagged, and
immediately merged back into develop. When one of the mid-day fixes caused a regression, the rollback was
a single step back to the previous tag, destroying zero other shipped work. Batching the fixes into one
release would have forced an all-or-nothing revert or a surgical diff-hunt under pressure.
THE RULE: Version granularity is your blast radius - make your tags as small as your tickets.
CHECK MY CODE, then report PASS / FAIL with file:line for each:
  1. Any hotfix branch that carries more than one ticket's changes (look for hotfix/ branches with commits touching unrelated files or referencing multiple ticket IDs).
  2. Any hotfix merge to the primary production branch that is NOT followed by a corresponding merge back into the development integration branch (develop never receives the fix or the version bump).
  3. Any release tag that bundles a net-new feature or new page alongside a behavior-changing fix (greenfield riding the hotfix train instead of waiting for the next minor).
THEN PRINT: a table (check, PASS/FAIL, evidence, fix) + a verdict (applies / partially / OUT_OF_SCOPE / no) + the single most important next action.