Every push to our internal-api repo showed a red failure, so we assumed CI was running and just unhappy with our code. That assumption was wrong twice over. The red came from a workflow we had deleted days earlier, one GitHub keeps firing forever because the registration outlives the .yml. A failed Actions payment on the org was firing its own identical red on top of that. And underneath all of it, the workflows we actually cared about had never run on a single one of our pushes. “Builds clean locally” had never once been checked by CI, and the angry red badge was the exact thing keeping us from noticing.
This is the kind of bug where unrelated facts conspire to produce one confident, false signal. Here is how it came apart.
The 0-second failure that wasn’t ours
The symptom was consistent. Every push produced a run that failed in 0 seconds, with a startup_failure conclusion, before any job logged a single line. We had been pushing Phase 0/A feature branches across two repos all day, the legacy-application-plus-SPA SaaS monorepo and the modern internal-api (a Hono service), and every one of them came back wearing the same red mark.
The natural read is “our CI config is broken.” So we went to look at the run, and the run pointed at a workflow we did not recognize:
gh api repos/<org>/<repo>/actions/workflows/<workflow-id>{"state":"deleted","path":"BuildFailed","name":""}Empty name. Path of BuildFailed. State deleted. This was a workflow file we had removed days earlier, and GitHub had kept its registration alive. Deleting the .yml does not delete the workflow as far as the Actions service is concerned. The registration sticks around with state: deleted, and it keeps firing a 0-second startup failure on every push that matches its old trigger.
Running gh run view on what looked like a “real” failure showed the same ghost id staring back. And these runs are not retriable. Trying to re-run one returns:
run ... cannot be rerunThere is no gh api call to permanently remove a deleted workflow. We checked. The only way to make a ghost like this stop is to contact GitHub Support and ask them to purge the registration. You cannot self-serve it away. So every push will keep getting decorated with a cosmetic red failure until a human at GitHub clears it.
That alone is worth knowing. The registration ghost also hid a bigger problem.
The billing block that muddied everything
Before we got to the finding, we spent a stretch convinced the ghost diagnosis was wrong and that this was purely a billing problem. That is worth naming, because both things were true at once and the overlap is exactly what makes a bug like this hard.
The same org had a separate Actions problem. It is on a grandfathered legacy GitHub Bronze plan that does not run Actions on private repos at all, and on top of that the org’s Actions payment had failed or hit its spending limit. GitHub kills a run before it executes when the org can’t pay for it, which produces the same 0-second startup_failure with no logs that the ghost produces. So midway through we flipped: “this isn’t a deleted workflow, every push is just getting killed at the gate for billing, and the prior session’s ghost call was wrong.”
Then we reconciled. It was both. The ghost workflow is real and fires its startup failure on every push regardless of billing. The billing block is also real and would kill the real workflows even if the ghost were gone. Two separate mechanisms, each independently capable of producing an identical 0-second red, stacked on top of each other. That is why it took a flip-flop to untangle: fixing one would not have explained the other, and the 0-second startup_failure they both emit is indistinguishable at a glance.
Nothing was triggering anyway
Once we stopped treating the red as meaningful and went to read the workflow definitions themselves, the floor dropped out.
The actual workflows in that repo, the ones we assumed were guarding our pushes, were CodeQL, lint, gitleaks, and npm-audit. We opened them and looked at the triggers. They gate on this:
on: push: branches: [main, preview] pull_request: branches: [main, preview]main and preview. That is it. We had been pushing feature branches and coalescing to develop all day, the Phase 0/A work across the SaaS and internal-api repos. None of those pushes were to main or preview, and we were not opening pull requests for them either. So every feature-branch and develop push triggered exactly nothing. No CodeQL, no lint, no gitleaks, no npm-audit.
Which means the validation we thought we had was a fiction. The eight held feature branches we had queued up to ship had never been touched by CI. “Build clean locally” was the only check that had ever run on them, and “build clean locally” is not CI. It is one machine, with its own node_modules and its own environment, declaring victory.
The red badge made this worse, not better. If those pushes had shown nothing at all, a blank Actions tab might have prompted the question “wait, is anything running?” Instead the ghost painted every push red, and red reads as “CI ran and found something,” which is the opposite of the truth. The noise impersonated a signal, and the signal it impersonated was the reassuring one: that there was a CI gate here at all.
The call
The right move once we understood the state was to stop. We had been about to bulk-ship five more feature branches to preview, each one about to carry the same startup-failure decoration and, more importantly, none of them validated by anything real. Shipping five more startup-failure-stamped branches would have buried the problem deeper and trained us to ignore red entirely, which is the worst possible habit to build around a CI surface.
So we paused the bulk ship until at least one branch could pass CI cleanly. The cleanest way to force the real workflows to fire here is to open a PR, since they trigger on pull_request against [main, preview] and a PR can’t be masked the way a direct push can. Then confirm the run that comes back is a real job and not the ghost id, and only then trust the green. A green you can trace to a real job is worth something. A red from an orphaned registration is worth nothing, and a silence from a branch your triggers exclude is worth less than nothing because it looks like safety.
The whole episode cost us a false sense of safety on eight branches and a flip-flop between two causes that emit the same 0-second red. The fix was learning to distrust the badge and go read the trigger.
Takeaway
Before you trust a CI result, read the branches: list in the workflow triggers to confirm the branch you pushed actually runs anything, and check that the run’s workflow id belongs to a live workflow and not a state: deleted registration with an empty name. If the branch is excluded and no PR is open, “it passed” means nothing ran. A deleted registration keeps firing a 0-second startup_failure and can only be removed by GitHub Support.
Related
- Our grandfathered $25 GitHub plan silently blocked every CI run, and the cheaper fix was a one-way door: the billing layer that was stacked on top of the ghost, emitting the same 0-second failure
- One Private Dependency, Five Different Failures: multiple independent failure modes producing a single misleading signal across deploy targets
- One monorepo, two build lanes: keeping classic-ASP pushes at zero CI minutes: designing CI triggers so the right branches run the right jobs
- UTC logs, a local clock, and the canary request: timezone discipline in an incident: reading observability signals carefully before acting on them
- Three compounding bugs spawned a burst of near-identical tickets, and tail -15 hid the evidence: another case where the visible signal obscured the real failure beneath it