123 sessions, 6 repos, one day

On Aug 18 the work log showed 123 sessions across 6 repos in a single day. The same day we cut one working queue from 33 to 18 items. Elsewhere, the ticket-creation default had put 34 items into a working queue that nobody chose.

Coordination is what does not scale for free. Every agent we add brings capacity, and also another party that can collide with the others over the same files, the same connector, the same archive. We saw it on Aug 17. We migrated ten infrastructure tickets out of the wrong tracker and closed the originals. The next morning 3 of them had been reopened by a session that was still running, and 9 new findings had landed in the wrong place too.

That is an old problem in distributed systems, and it is the reason locks, queues and transactions exist.

The attention it takes to keep agents from stepping on each other does not shrink on its own. It shrinks when you build something to replace the human coordinator. We shipped a guard that refuses tickets of the wrong kind at creation time, and the leak still continued overnight in the session that was already running. Until something better exists, one of us does that job.

Takeaway

Each agent you add can collide with the others over the same files, connectors and archives. In our case, a guard at creation time did not stop a session that was already running, and a person still did the coordinating. Ask who or what coordinates your agents’ shared resources, and whether a running session is covered by the same guard as a new one.