---
title: "Three Blank Fields and a Tool Nobody Thought to Check"
canonical: https://dxdev.com/everyday-ai/shipped-is-not-the-same-as-running/
datePublished: 2026-08-14
---
A screenshot arrived with one line under it. A [task kept skipping the current work period](/blog/the-automation-that-never-ran/). The person it should have been assigned to was blank. And if there was a version number involved, that was blank too. Not once in a while. Every single time.

We had a tool built specifically to fill in those three details automatically the moment a task got marked done. It had been sitting in the system for a while. Nobody had complained about it, so we assumed it was doing its job in the background, the way a good automatic step should.

It had never once worked. Not on that task, not on any task, not for anyone, since the day it was turned on.

## Careful logic that never got to run

The logic inside that tool was well thought out. It knew how to tell the difference between everywhere a task had ever passed through and where it currently sat. It handled the messy cases correctly. Somebody had clearly sat down and solved the problem.

And it had never run. The logic was not wrong. Something earlier in the chain was broken in a way that stopped the whole thing before it reached the good part. One small setup error at the very front of the tool crashed it before a single line of the careful, correct logic underneath had a chance to fire.

Once that first crash was fixed, we found four more places, stacked one after another, where a single missing piece could have killed the whole thing again without a sound. Any one of the five would have produced the same result: nothing happens, and nothing tells you that nothing happened.

## Silence looked exactly like success

A tool gets built, it gets tested once, the test passes, and it ships. From then on, the only signal anyone has that it's still working is silence. No error messages, no complaints, nothing flagged.

Silence is what you'd see whether the tool was working perfectly or had been broken since day one. The two situations look identical from the outside. The only way to tell them apart is to go check a real, recent case and see whether the thing that was supposed to happen actually happened.

We keep three details on every closed task, and all three had been broken the whole time. We found out from the same screenshot, on the same day, because we happened to be looking at that particular task closely enough to notice.

## What the AI assistant did, and what it didn't

This is the kind of checking an AI assistant is good at, if you point it at the right question. Not "does this look like it's set up correctly," but "did this actually happen on a real, recent case." An assistant can trace what a step was supposed to do, pull a handful of recent examples, and tell you plainly whether the expected result shows up.

What it shouldn't do is turn anything on or off, or decide on its own that a fix is safe to apply. Its job is to surface the gap between "this was built to happen" and "this actually happened," and hand that back to a person to act on.

## What a person still has to decide

A person has to decide what proof looks like for each automated step that matters, and how often it's worth checking. Some things are worth a weekly glance. Anything touching money, customer records, or a deadline deserves a standing check that flags itself the moment it goes quiet, rather than waiting for someone to happen to notice.

A tool passing its first test and getting turned on proves it worked once. It does not prove it has worked since. For each automated step that matters to your business, pick one recent, real case and confirm the expected result actually happened.