--- title: "When Parallel Sessions Overload: Building a Spawn Governor" canonical: https://dxdev.com/blog/spawn-governor-load-control/ datePublished: 2026-09-15 --- The CI-alert triage job died at 2:41 AM: "You've hit your session limit ยท resets 5:30am (America/New_York)." At 3:40 AM the same message came from a completely different session. At 4:10 AM, a third. Three unrelated jobs, one shared seat, all dead within ninety minutes, all waiting on the same reset clock. Whatever was consuming that quota wasn't three sessions running normal work. It was something spawning far more of them than the seat could hold. The session I eventually used to run this down started at 1:23 AM and didn't close until 4:26 PM. Fifteen hours and change, on one ticket: the recurring workstation overload problem. That's the honest shape of a root cause hunt that doesn't go in a straight line. ## The gate as the first fix, and why it wasn't one The obvious lever, once the overload was visible, was the decision gate, the permission layer that role-scoped agents (window-manager, window-agent, auth-manager) have to clear before touching anything privileged, credentials, host lifecycle, self-authorization. Tighten the gate, the theory went, and the runaway spawning stops because the dangerous actions at the bottom of the chain get refused before they do damage. That's what happened next, and it's visible in the log as a wall of REFUSED-BY-GATE entries: auth-manager refused at 5:29 PM. window-manager refused at 7:01 PM for touching its own authorization surface. auth-manager refused twice more at 7:14 PM, once at the Bash tool layer, once at the gate itself. window-manager declined at 7:33 PM, then got refused again a minute later. window-agent got refused three times in a row between 7:49 and 7:51 PM. window-manager got refused again at 8:06 PM for a host-lifecycle call, taking a machine down or destroying one. Nine refusals in under three hours, each one a fresh haiku-model session spun up, each one burning real quota to get told no. The gate was doing its job, every dangerous call it saw, it stopped. But it wasn't fixing the overload. It was proof the overload was still happening: something kept spawning new attempts at privileged actions, and hardening the gate just meant each attempt died a little faster and a little more expensively than before. The refusals weren't the fix. They were the symptom, restated nine times. ## What the governor actually had to do The gate operates per call: is this specific action allowed for this specific role. It has no concept of "how many of these have been tried in the last hour" or "how many children has this session already spawned." That's a different layer, and we hadn't built it yet. The actual fix was a live spawn budget: a cap enforced at the point where a session tries to create another session, not at the point where the child tries to do something dangerous. Paired with it, a test-lock governor, so that parallel sessions couldn't pile onto the same test resources and retry-storm each other into the same quota wall that killed the CI-alert triage job, the HQ automation job, and the weekly bizops job within the same ninety minutes that morning. Cap the spawning at the source, and the flood of gate refusals downstream mostly stops needing to happen at all. Building the governor surfaced a second problem: some of the existing safety guards had quietly broken. The platform guard, which is supposed to stop client-breaking changes from going out unreviewed, wasn't actually enforcing anything, it had degraded silently, with no log entry to show it had stopped working. That's arguably worse than the overload itself: a guard that fails open and doesn't tell you. We restored it with logging, so the next time it refuses something, or the next time it silently stops enforcing, there's a record either way. The partner-push approval path got hardened the same night, and about twenty stale or duplicate tickets that had accumulated during the overload got closed out, since a chunk of them were the same underlying spawn-overload symptom filed under different tickets by different sessions that didn't know about each other. ## The review cadence around it None of this shipped on one person's say-so. Through the same stretch, a second review agent kept running independent verdicts on the proposed changes, high-effort review passes returning "FIX-FIRST" over and over: 4:48 PM, 4:52 PM, 5:49 PM, 6:57 PM, 7:22 PM, 7:59 PM, into the evening. FIX-FIRST isn't a rubber stamp, it's a verdict that says the change is right in direction but not safe to ship as written yet. It took until 10:07 PM, a separate medium-effort pass, for the verdict to flip to SHIP. That gap between the first FIX-FIRST and the eventual SHIP is where the actual engineering happened, closing the gap between "the gate is refusing correctly" and "the thing generating bad calls in the first place has a cap on it." The gate was never the wrong layer to have. It was just the wrong layer to expect a fix from. A permission check answers "should this one call happen." A budget answers "how many of these get to exist at once." Overload problems live in the second question, and no amount of hardening the first one gets you there.