51.8K to 14.9K and the Markdown Still Rendered Fine

One regenerated file dropped from 51.8K to 14.9K, and that shrink was the only thing that kept a refactored extractor from shipping. We had dropped every tool-output record from the file. The markdown still rendered cleanly enough to fool us.

What the extractor does

It transforms conversation-like records into two versioned artifacts: a structured form for downstream processing and a human-readable view for review.

The original code built those artifacts along separate paths, which made divergence more likely. The refactor moved both renderers onto a shared intermediate representation. That cut duplication, but it did not guarantee correctness. A shared representation can still omit, transform, or leak data, so it needs an explicit output contract and independent checks.

One helper, two jobs

The extractor had a classification helper with a legitimate job. It separated records that should count toward a content-quality threshold from records that were protocol or tool plumbing. That classification also informed a human-readable label.

Those are scoped questions: what counts for selection, and which content informs a label. Neither is a question about whether a record should exist in the retained source or a published artifact.

So the triviality check is correct for what it’s for. The mistake was reaching for it during the refactor to gate something else entirely.

During the refactor, we reused that helper to gate row construction. It answered a nearby but different question, and a whole class of records disappeared from both generated artifacts.

Nothing broke. The output stayed well-formed, which was the trap, because a render that succeeds says nothing about whether it is complete. A downstream process would have received a materially incomplete record set unless the regression checks made the omission visible.

The tell was a size delta

The first signal was a large divergence between a regenerated artifact and its approved fixture. A big size change in an output that should stay comparable does not prove a defect, but it is a high-priority review signal.

The versioned fixture made the behavior change visible before we understood its cause. Diffs, checksums, and size thresholds work as tripwires once their limits are explicit. They need deterministic or normalized inputs, and they have to be paired with semantic expectations and human review of intentional changes.

Splitting the gate

The fix was to restore separate contracts. Selection logic answers whether a record counts toward a quality threshold. Row construction answers whether the approved output contract requires that record. Tests now cover both decisions independently.

The regenerated fixtures then matched the approved baseline after normalizing any intended non-determinism. That gave us a stable starting point for future intentional changes. It does not replace tests for record completeness, schema validity, or downstream behavior.

The comparison also prompted a broader privacy review. Metadata and frontmatter were generated on a different path from the main body, so they needed the same privacy and validation controls. Every emitted surface (metadata, manifests, sidecars, logs, exports, and error reports) needs an explicit classification and appropriate handling.

What to keep

A file that renders cleanly can still be missing a whole class of records, so pair the fixture comparison with checks for record completeness and schema validity.

dxdev LESSON · paste into your AI coding agent
LESSON: Committed Output Is a Free Golden Test
SOURCE: dxdev.com/blog/committed-output-is-a-free-golden-test
WHAT HAPPENED: A refactor reused a classification helper to decide whether records existed in generated output. The result rendered without errors but omitted an entire record class. A large change against a known-good committed fixture surfaced the difference; targeted semantic assertions then established what was missing.
THE RULE: When code regenerates versioned artifacts, compare against approved fixtures when generation is deterministic, define the expected semantic invariants, review every intentional delta, and run privacy and provenance checks across every emitted surface.
CHECK MY CODE, then report PASS / FAIL with file:line for each:
  1. Any helper whose classification purpose is being reused as an existence, authorization, redaction, or persistence decision without an explicit contract and tests for both behaviors.
  2. Any generated artifact lacking deterministic-build controls, approved fixtures, semantic invariant tests, and a review path for intentional changes.
  3. Any emitted surface-body content, metadata, manifests, sidecars, logs, exports, or error reports-that bypasses the appropriate privacy, provenance, and validation controls.
THEN PRINT: a table (check, PASS/FAIL, evidence, fix) + a verdict (applies / partially / OUT_OF_SCOPE / no) + the single most important next action.