I refactored an extractor, the output looked fine, and I almost shipped it. The only thing that stopped me was one regenerated file dropping from 51.8K to 14.9K. That shrink was the whole alarm. I’d silently dropped every tool-output record from the file, and the markdown still rendered cleanly enough to fool me.

The thing I was building

I was refining an extractor that transforms conversation-like records into two versioned artifacts: a structured form for downstream processing and a human-readable view for review.

The original code built those artifacts along separate paths, which made divergence more likely. The refactor moved both renderers to a shared intermediate representation. That reduced duplication, but it did not guarantee correctness. A shared representation can still omit, transform, or leak data incorrectly, so it needs an explicit output contract and independent checks.

The gate that was doing two jobs

The extractor included a classification helper whose legitimate job was to distinguish records that should count toward a content-quality threshold from records that were protocol or tool plumbing. That classification also informed a human-readable label.

Those were scoped questions: what should count for selection, and which content should inform a label. They were not questions about whether a record should exist in the retained source or a published artifact.

So the triviality check is correct, for what it’s for. The mistake was reaching for it during the refactor to gate something else entirely.

During the refactor, that classification helper was reused to gate row construction. It answered a nearby but different question, so a class of records disappeared from both generated artifacts.

The rendered output remained well-formed. That was the trap: render success does not establish completeness. A downstream process would have received a materially incomplete record set unless the regression checks made the omission visible.

The tell was a size delta

The first signal was a large divergence between a regenerated artifact and its approved fixture. A substantial size change in an output expected to remain comparable was not proof of a defect, but it was a high-priority review signal.

The versioned fixture made the behavior change visible before its cause was understood. Diffs, checksums, and size thresholds are useful tripwires when their limits are explicit: they require deterministic or normalized inputs, and they must be paired with semantic expectations and human review of intentional changes.

The fix: split the gate

The fix was to restore separate contracts: selection logic answered whether a record counted toward a quality threshold, while row construction answered whether the approved output contract required that record. Tests covered both decisions independently.

The regenerated fixtures then matched the approved baseline after normalizing any intended non-determinism. That established a stable starting point for future intentional changes; it did not replace tests of record completeness, schema validity, or downstream behavior.

The comparison also prompted a broader privacy review. Metadata and frontmatter were generated on a different path from the main body, so they needed the same privacy and validation controls. Every emitted surface-including metadata, manifests, sidecars, logs, exports, and error reports-needs an explicit classification and appropriate handling.