Four tests failed on the Linux side of our gate, and only there. One of them was the session index’s same-size status flip test, and we made it pass by setting the file’s mtime forward.
The index is the session manifest index from step 1 of a larger piece of work. Its refresh() walks the tree with os.scandir instead of rglob plus stat. On Windows DirEntry.stat is free, and the whole 4,780-file tree takes about 25 ms. It deliberately ignores directory mtimes, because an in-place status flip does not change them.
The green we shipped
The test flips a manifest’s status without changing its size. On the Linux box it failed. The commit that greened the gate handled it like the other three failures. The SessionStart budget fixture got a detect_remote stub, the presence-ledger sampler test was skipped off win32, and the reboot-cli test asserted pythonw only on win32. For the session index test, the commit’s own line reads “bumps mtime explicitly (coarse Linux time…”, and the text cuts off there in my log.
What the reviewers saw
Three review runs flagged the same test file, tests/test_session_index.py, on that commit:
manus, rated high: advancing mtime “makes the test pass while hiding the actual cache-correctness bug.”
codex, rated medium: “Forcing a later mtime hides the production bug exposed by Linux: refresh() compares only size…”
codex again, rated medium, about twenty minutes later: “Advancing the mtime hides the production defect exposed by Linux: refresh() trusts unchanged …”
Both codex lines are cut off in the log, so I can’t quote what they said refresh() trusts. The fix commit says it plainly: the index trusted same-size, same-mtime metadata for a manifest it had just parsed.
That changed what the Linux failure meant. All three reviews said the failure exposed a production bug and that my mtime bump made the test pass while hiding it. A follow-up commit of review fixes for the same change followed.
The fix
The fix made the index stop trusting same-size, same-mtime metadata for a freshly parsed manifest. The commit calls this a racy-timestamp reparse.
The regression test goes the other way from the one I had edited. It keeps the metadata unchanged, and it fails without the production fix. The same commit also changed the reboot_cli test to follow production’s pythonw existence check. test_session_index and test_reboot_cli pass together, 25 tests in all.
AI Skills
Use this lesson with the AI assistant you already use
Four tests failed on the Linux side of a gate, including a session index test that flips a manifest's status without changing its file size. The commit that greened the gate bumped the file's mtime forward, and three separate review runs flagged that change as hiding a real cache-correctness bug.
Paste the prompt, share only the context needed to answer it, and treat the result as a draft for your review. Do not include confidential information or let an AI assistant make changes without your approval.
The prompt works in any AI assistant as it is. We are also building a /dxdev skill for visual reports and saved memory.
It is not released yet: see what you can use today.
dxdev LESSON · paste into your AI coding agent
LESSON: A Test That Only Passes After You Nudge Its Inputs Is Hiding a Production Bug
SOURCE: dxdev.com/blog/seeded-test-mtime-hid-racy-cache-bug
WHAT HAPPENED: The session index's refresh() skipped re-reading a manifest when size and mtime looked unchanged. On Linux, coarse timestamp resolution meant a same-size status flip could leave the mtime identical, so the test failed. Instead of asking why, the author set the mtime explicitly so the test would pass, and the gate went green. Reviewers from manus and codex, three runs in all, each noted that this made the test pass while concealing a production defect. The index was trusting same-size, same-mtime metadata for a manifest it had just parsed. The fix made the index reparse in that racy-timestamp case, and the new regression test keeps the metadata unchanged so it fails without the production fix.
THE RULE: When a test fails only on one platform, treat the failure as possible evidence of a production bug before editing the test environment to make it pass. A regression test should reproduce the failing condition exactly, not adjust it away.
CHECK MY CODE, then report PASS or FAIL with file:line for each:
1. For every test that sets or advances a timestamp, size, or other input metadata to make it pass, there is a written reason that the production code is not relying on that metadata being distinct.
2. Each platform-only test failure has a recorded diagnosis of whether production code or the test is wrong, rather than just a skip, stub, or input tweak in the commit that greens the gate.
3. Each regression test for a cache or change-detection bug leaves the metadata unchanged and is confirmed to fail when the production fix is reverted.
THEN PRINT: a table (check, PASS/FAIL, evidence, fix) + a verdict (applies / partially / OUT_OF_SCOPE / no) + the single most important next action.
