One Thursday, by 12:21, I had shipped two production hotfixes and merged a third change to a preview branch. Three changes, two lanes, and nobody told me which lane each one belonged in. I didn’t decide by diff size, release window, or which one I felt like doing first. I decided by what each change could break if I got it wrong.
That’s the whole heuristic, and it’s boring on purpose: route a change by its blast radius. A production-affecting bug that you can verify gets a hotfix and ships now. An internal refactor with no user-visible behavior rides the preview branch and goes out in the normal release. Once you say it out loud it sounds obvious. In practice people route by size (“it’s only four lines, just push it”) or by schedule (“we ship Fridays”) and both of those get you in trouble.
Here’s the morning, concretely.
The two hotfixes
Change one was a Classic-ASP redirect bug. The tournament bracket page was throwing our internal-error screen, the dreaded “yellow box,” whenever it hit a missing or bad bracket ID. The page was building a URL out of query-string fragments and handing it to Response.Redirect. Response.Redirect issues a 302 by writing a Location header, and if the response buffer is already in flight, that throws. Our error-handling wrapper catches the throw, logs to the error table, and transfers to the internal-error page. So a missing ID, which should have been a graceful bounce, detonated into a full error screen for the user.
The fix was four lines in, twenty-four lines out: swap both Response.Redirect calls for a thin Server.Transfer wrapper. Server.Transfer re-routes execution server-side with no new HTTP round trip and no header dependency, so it physically can’t hit the “headers already sent” failure mode. I also left an honest // PENDING in the code flagging that the home page it falls back to still loads with its own bugs. I treated the acute symptom and marked the deferred root cause instead of pretending it was clean.
This was user-facing, in production, and I could reproduce and verify it. Hotfix. It went out as a tagged release at 09:59.
Change two surfaced a couple of hours later, and nothing in our code had changed. The “copy embed code” button quietly stopped working for some users. The cause was an aging library and a brand-new browser API fighting over the same global variable name.
We ship an older build of clipboard.js. Its UMD wrapper assigns the constructor to window.Clipboard. Our code did new Clipboard("#copy_button"). Fine for years. Then modern browsers shipped the Async Clipboard API, which puts a native, non-constructible Clipboard object on window. Depending on load order and browser, window.Clipboard is now the native object, and new Clipboard(...) throws because you can’t new it. Silent copy failure, no error anyone reported, just a button that does nothing.
I didn’t upgrade the library. An old minified blob loaded on a lot of pages, in an app with no module bundler, is a bad thing to bump for a single bug. The lower-risk move is a defensive shim at the call site: feature-detect which constructor you actually have (window.ClipboardJS || window.Clipboard), check it’s really a function, try/catch the construction, and degrade. The fallback chain ends at navigator.clipboard.writeText() for the native path, then document.execCommand('copy') against an offscreen textarea for the truly old browsers.
User-facing, in production, reproducible, verifiable. Hotfix. It went out tagged at 12:21.
The refactor that didn’t get a hotfix
The third change was the biggest of the three by a wide margin: about 760 lines across two coupon-admin pages. The before was 1990s string-soup, <%= %> interpolation tangled into hand-written table and form HTML, driven by an ADO recordset loop with the usual .MoveNext() and .EOF lifecycle. I rewrote both pages into a fluent DOM builder on the page object, where each element returns this so the page reads as a tree, and swapped the recordset loop for a plain array query. Two correctness wins fell out for free: every value now runs through an escape function instead of being dumped raw into HTML, and the empty state actually renders something instead of nothing.
By line count this was the heaviest change of the day. By blast radius it was the lightest. It’s an internal admin page, no user-visible behavior change, and it’s exactly the kind of structural rewrite that benefits from a second set of eyes before it touches a release. So it merged to the preview branch with no version tag, where it can sit and get looked at before riding out in the normal release like everything else.
That’s the part people get backwards. The instinct is to fast-track small changes and be careful with big ones. Size is the wrong axis. The four-line redirect fix was riskier to defer than the 760-line refactor, because the four-liner was actively throwing error screens at real users and the 760-liner wasn’t changing anything a customer could see. The question is what happens in production if you’re wrong, and how fast you’d know.
Versioning isn’t ceremony here. The version tag is a signal about blast radius. A tagged X.Y.Z hotfix means “this touched production behavior, here’s the marker.” No tag on a preview branch means “this is internal, it goes through the normal gate.” The lane and the version are saying the same thing.
Count the work, not the clones
There’s a postscript to that morning that’s worth its own warning. The raw activity number for the day looked impressive. It wasn’t.
The actual intellectual work was three changes: one redirect fix, one namespace-collision shim, one page refactor. The big commit count came from the topology. We run a fleet of clones, and every merge, tag, and back-merge replays across all of them. The same handful of merges and tags landing across a dozen places gets you a number that has almost nothing to do with how much thinking happened.
I bring this up because the commit count is a seductive metric and it lies in exactly this situation. Fan-out inflates the count without inflating the thinking, and measuring anyone by commits in a multi-target setup rewards the topology, not the work. The honest measure that day was three.
Related
- The Migration Flip That Took Down a Live Customer (and the Blast Radius I Could Measure in 12 Seconds): what happens when blast radius is underestimated before a change ships
- Seven production releases in one day: the case for tiny, tagged, revertable hotfixes: the same discipline applied to release cadence
- The omnibus ticket is a coordination tax: splitting one QA pass into six: decomposing work by scope so routing decisions stay clear
- Response.Redirect Blew Up the Error Page. Server.Transfer Didn’t.: the yellow-box hotfix referenced in this post
- The Cheapest Fix to My Backlog Was Splitting One Status Into Two Words: another decision about routing work that looks simple but has blast-radius consequences