A Wall Street Journal profile landed in our group chat this morning: a Sausalito company with ten thousand paying customers, on track for eight figures, zero employees, AI answering the email and writing the code. The reaction split the way it always splits. Half the readers are sure it is a fantasy, half are sure it is inevitable, and almost nobody asks the operational question, which is what happens on the day the AI gets something wrong.

We have been running a version of this experiment for months, on a legacy platform with real customers and real money moving through it, driven by a fleet of Claude Code sessions rather than one assistant. Our version doesn’t have to be reconstructed from memory for a journalist. Here is one recent stretch, straight from the ledger:

 55  review verdicts produced by parallel agent sessions
 51  of those refuted by an adversarial audit pass
136  rejected emails joined back to customer orders
     (99 internal batch mail, 37 customer-facing, ~25 addresses)
  2  apology drafts written, 0 sent (sending is not permitted)
  1  money-move ask from the wrong party: one reply, then quarantine
  5  guardrail hooks found failing open (missing dependency)
  4  minutes of production slowdown, caused by one of my own agents

Every line is checkable against a commit, a ticket, or a work log. Keep the table in mind while we walk through what the mainstream story gets right and what it cannot afford to print.

What the profile gets right

You really can run meaningful chunks of a business through AI agents now: triage, first-pass code, first-pass customer replies, the work that used to justify a first hire. We measured six months of that shift in our own git history and published the numbers, so I won’t re-argue it here. The profiles get one thing right without saying it out loud: every one of these companies built infrastructure around the prompting, and that infrastructure is the story too boring to profile.

What the profile skips

Most of our work now goes into the permission layer, the part that decides what an agent may do without anyone in the room.

Two examples from the table. The quarantine line: a party in a shared Discord asked our agent for a money move they had no standing to request. The agent’s old behavior was to refuse politely every time the ask got rephrased, which meant a determined asker could just keep rephrasing. Its new behavior is one reply, then silence. The person and their own AI are treated as a single principal, and the quarantine can be lifted only from a control surface they cannot reach.

The apology line: after a week of bounced email, an agent pulled the raw mail logs, joined 136 rejected sends back to real orders, worked out that expiry warnings were the biggest casualty, and drafted the two customer apology emails, flagging eight accounts that deserved a personal note instead of a form letter. Then it stopped, because drafting is inside its authority and sending is outside it. Before we acted on any of it, a second session, given only the raw logs, recomputed the same split. An apology sent on our behalf to a customer carries a different kind of risk than a draft sitting in a queue, so the queue is where it waits.

The profile treats the AI as a single actor that writes the code and answers the email. What actually runs is closer to a small bureaucracy: one agent proposes, another checks, and a third holds the rules about which customer, which account, which part of the codebase the first two may touch at all.

The verification tax

Here is the tax the profiles never price in. Every claim an agent makes about its own work is a hypothesis until something outside the agent confirms it. The 55-verdicts line in the table is what happened when we forgot that. We fanned a complex review out to parallel sessions and got back confident, well-written verdicts. An adversarial audit pass then refuted 51 of the 55. The wrong verdicts read exactly as confident as the right ones, and confidence was the only thing the transcripts showed.

So the operating rule underneath everything else: an agent’s report on itself is not evidence. Evidence is the commit, the log line, the customer’s actual reply. Paying for that check on every cycle is a real line item, and the full version of that story would print it.

What actually breaks

Mostly nothing dramatic. The worst incident so far is on the table: an autonomous session ran an unbounded scan against the production database and slowed the live site for four minutes. The guards followed immediately: machine-level blocks on unbounded queries, a bounded wrapper with timeouts, and alerting that treats a database stall like a CPU spike.

The less visible failures worry us more. Five of our guardrail hooks spent weeks failing open because a dependency was missing and they defaulted to allow whenever their check could not run. The fix took minutes. Noticing took weeks, because a guard that fails open produces no signal at all.

Why write this down now

Because the story has gone mainstream, someone is about to read that profile, spin up a few assistants over a weekend, and skip straight past the boring 90 percent: the quarantine logic, the audit passes, the line between draft and send. The AI answering your email really is the easy part.

Somewhere in our queue tonight there are two apology emails, addressed, reviewed, and ready, that nothing in the system is allowed to send. They will still be there in the morning, waiting for us. That wait is the part that never makes the profile.

Three checks before you hand an agent customers

Before relying on AI agents for customer-facing work, check three things. First, what each agent may send versus only draft. Second, who or what verifies an agent’s claims about its own work (an audit pass refuted 51 of 55 review verdicts here). Third, whether any safeguard defaults to allow when its check cannot run. Five of our guardrail hooks did exactly that for weeks and produced no signal.