For weeks we had a theory: a phone connection was dropping, and that was killing our work. Every time it happened we nodded along to our own explanation and asked for a fix aimed at the connection.

The fix we asked for was not the fix we needed. The thing killing our work was not the connection at all. It was restarts we were triggering ourselves, on our own computer.

What the system’s own record said

Instead of accepting the theory, we pulled the system’s own record and lined it up against what had actually happened. Two things came out of that right away.

Work that had been running overnight with nobody connected to it had survived the whole night untouched. That ruled out the connection as a cause. If a dropped connection were the killer, overnight work with nothing attached to it should have died too. It didn’t.

What did kill things was restarts we had done ourselves, of the computer and of the program hosting the work. Everything tied to that program died at the exact same moment every time. That is what it looks like when a shared parent goes down. Random crashes would have been scattered across different times.

There was a second mistake sitting on top of the first. We had also written off several pieces of work as failed, because they had gone quiet and stopped producing new output. When we checked properly, all of them were still running fine. They were waiting, and one of them was paused mid-task until we said the word to continue.

That misreading nearly sent us toward the wrong fix a second time. Because we believed the connection was the problem, the first idea on the table was a tool built specifically to survive dropped connections. It would have solved a problem we did not have.

The fix, once the cause was known

Once the cause was confirmed, the fix was small and specific. We built a way to move a piece of work that is already running over to a phone without losing its history. If the computer does need a restart, getting back to where we were now costs one request instead of a full reconstruction. That fix only made sense after the connection theory was ruled out. Built around the wrong cause, it would have solved nothing.

What we check now

Before we call something dead, we check whether it is still running, not just whether it has gone quiet. Before we commit to a fix, we look at what the record says caused the problem instead of running with the first explanation that felt familiar.

The restarts were ours the whole time. We would never have found that by staring harder at the connection.