When One System Does Two Jobs: Verifying the Fast Path
At 11:36 AM on June 29, one click on Send Domain Instructions Email was taking the long way through our mail system.
The build log
Notes from running agents in production.
Build log, architecture patterns, and observations from running autonomous AI systems in production.
-
-
The 200-Line Lie: Why My Agent Kept Forgetting My Goals
I kept saving goals to my agent's memory and they kept vanishing by the next session. The cause wasn't a flaky model. It was a silent truncation cap, and the fix is a pattern anyone running long-lived AI should steal.
-
Before you repair data, build the diagnostic
Many apparent software failures are mismatches between the story a user reports, the state a system stores, and the assumptions a repair would make. A focused diagnostic can separate genuine drift from valid but unfamiliar data before anything is changed.
-
The Zombie That Blocked Everything
A single stuck proposal generated 17 cascading alerts and created a meta-deadlock. Here's how a zombie in the review queue taught us that every governance layer needs a fire exit.
-
The Auto-Heal Worker That Almost Broke Itself
At 2:54 p.m., the live production log showed the certificate auto-heal worker trying to process cf-heal-state.json as a command.
-
Infrastructure Self-Healing Without the Ops Team
At 4:10 PM, a migrated site had the right DNS records and no HTTPS.
-
Built the Review Tool, Then Told the Other Dev to Skip It
At 6:56 p.m., I had 12 unanswered design questions and a review tool I had built so another developer would not have to answer them in a ticket comment.
-
I stopped babysitting the browser, then had to build the part that was missing
At 10 concurrent agents, the browser stopped being a tool and became a scheduling problem.
-
The Internal Review Said Clean. A Second Pass Found Six Real Bugs.
A bulk-edit feature passed its own code review on SQL injection, access control, and transactions. Sending it to an independent second reviewer found a second call site the fix never covered, a silent data-clearing bug, and a scope bug the first pass never asked about.
-
Fixing Code You Can't Reproduce
At 3:36 PM, we shipped a release because one organization-account tournament event was stuck and the exact trigger would not come back on demand.
Follow the Build
The real failures and fixes from building AI systems, one practical lesson per post. Get the next one in your inbox.