The Bounce Breaker Tripped at Message 425 With 1,776 Emails Pending

The bounce breaker tripped at message 425 of a 500-message batch, and 1,776 win-back emails were still pending. It was a Wednesday. The schedule we were holding said finish Thursday, no Friday sends.

What the breaker saw

The win-back mailer was already going out through SES. Tuesday had two batches of 500. The ramp plan had Wednesday at three batches of 500, then Thursday at two more plus 201. Wednesday stopped at 425, so 1,425 more had gone out in the resumed run when the hold landed.

The breaker is simple. It is a flat count of 25 bounces in 24 hours, and once it trips the send stays held until a human releases it in the admin screen. It reads from the SES bounce receiver we had already put live. SES publishes to two SNS topics, both subscribed to one events endpoint, and the receiver is backed by two tables, dbo.sesEvents and dbo.sesSendHold. The breaker did its job. The plan did not survive contact with it. A schedule built on “500 a batch, three batches a day” had no term in it for a flat 24-hour bounce ceiling, and 1,776 messages sat in the queue against a Thursday finish.

I can’t tell you from the logs why the bounce count climbed. What the stall did was make the larger problem visible: this mail was growing up one campaign at a time, and nothing described where it was supposed to end up.

Reviewing the plan with the mailer paused

The migration strategy had a simple shape:

• Split mail into two buckets, company mail and customer mail.
• Give each bucket its own SES path.
• Retire PowerMTA and the old sending domain.

With the send held, I ran it past two reviewers, Manus and Codex. Codex took 94.6 seconds. Their replies produced a merged v2 and then a v3.

The first finding was a missing file

Manus’s first finding had nothing to do with SES. The plan named a canonical routing document as the authority for Phase 0, the inventory step. That file was not on the current master branch of the code repo.

A plan whose first instruction is “start from this file” fails at that instruction if the file isn’t there, and everything downstream inherits the gap. Step zero pointed at a document that was not where the plan said it was.

Codex went after a principle I had adopted as a requirement: one suppression list, checked by every send path. Its reply opened by telling me to revise it. The breaker that stopped the send was the newsletter one, which made the point hard to wave off. “Checked by every path” is a claim about every path, so it has to be argued per path and can’t be assumed from the existence of one table.

What the plan leaves open on purpose

The plan ended up with 32 ordered tasks in its seventh section. Phase 0 comes first, and the domain and identity for each bucket are explicitly left undecided until that inventory is done. Nothing is locked in.

Parked, with pointers

The plan now lives on a ticket with status backlog. The ticket links the original draft, the Manus merge, both review replies, a visual brief with its render script, and a query script that re-runs the volume numbers so they can be refreshed rather than trusted. The resume instruction is one sentence: review v3, then decide whether to create the migration epic with its 32 tasks.

I did not create the epic. Turning 32 tasks into scheduled work on the same day a send is held, with a deadline in play, is how a half-verified plan becomes a half-built migration.

The 1,776 were still queued behind the breaker when I parked it. Releasing the hold is a human decision, and so is the order of work after that. The one rule I took from the stall fits in a sentence: before you build on a document, check that it is on the branch you think it is.

dxdev LESSON · paste into your AI coding agent
LESSON: Check That The Document Your Plan Starts From Exists On The Branch You Think It Does Before Building On It
SOURCE: dxdev.com/blog/mailer-infrastructure-modernization
WHAT HAPPENED: The bounce breaker stopped Wednesday's send, and the ramp schedule had no term for a flat 24-hour bounce ceiling. While the mailer was paused, the author sent a migration plan (split company and customer mail, give each its own SES path, retire PowerMTA) to two reviewers. Manus found that the canonical routing document named as the authority for Phase 0 was missing from master, so the plan's first instruction pointed at nothing. Codex challenged the requirement of one suppression list checked by every send path, noting that a claim about every path has to be argued per path. The plan grew to 32 ordered tasks with domain and identity choices left open until the inventory is done. The author parked it on a backlog ticket with pointers and did not create the epic, to avoid turning a half-verified plan into a half-built migration.
THE RULE: Before you build on a document, confirm it exists on the branch you think it is on. A plan whose step zero cites a missing source inherits that gap in every later step.
CHECK MY CODE, then report PASS or FAIL with file:line for each:
      1. Every file or document a plan names as an authority or starting input resolves on the branch the plan will be executed against.
      2. Any claim of the form 'checked by every path' is verified separately for each send path rather than inferred from one shared table existing.
      3. Schedules that depend on a rate or volume ramp include an explicit term for hard ceilings such as a flat 24-hour bounce limit, and state who releases a held send.
THEN PRINT: a table (check, PASS/FAIL, evidence, fix) + a verdict (applies / partially / OUT_OF_SCOPE / no) + the single most important next action.