Why a Blank Importance Score Became 0.5 Instead of Zero

Four small bonuses sat in a row on the screen: 0.30, 0.25, 0.20 and 0.15. We had added them one at a time to help our note-finding system pick better. A note waiting more than five days got 0.30. Something due within a week got 0.25. An overdue follow-up got 0.20. A recently updated note got 0.15. The math was neat. The results were not.

The system’s job was to gather a small stack of useful notes whenever someone asked a question. That stack gave an AI the background it needed to answer well. We started by putting almost all the trust in cosine similarity, a way of measuring whether a question and a saved note are close in meaning. Then we added those four bonuses on top.

It sounded sensible. If a note matched the words in the question, it should rise. If it was also due soon or had been waiting around, it should rise a little more. But the close match got the entire starting score. Everything else was a sticker stuck on afterward.

That first version did not crash. It did not produce wild numbers. A test would have watched it run and passed it. The problem was what the formula believed without telling us: that a close text match mattered more than anything else, by default. We never meant to make that choice, and the formula made it for us anyway.

What it cost

The cost was time. The target for this first version was blunt: the bundles of notes had to be relevant more than 80% of the time in daily use. Until we got there, we could not move on to the more visible parts of the product. A ranking system that keeps handing over the wrong background makes the whole thing feel dumb, even when every screen looks polished.

Take a question about what needs attention now. An old note from a project that has been quiet for ages might use almost exactly the same words as the question. It could beat a slightly weaker match that is important, active and overdue. The only way the old formula could fix that was with yet another special bonus. Soon the ranking was a junk drawer full of little exceptions.

Splitting the starting score on purpose

The fix was a different starting score, not a fifth sticker. We made it shared deliberately. Sixty cents of every dollar went to how closely the note matched the question. Twenty cents went to importance. Fifteen cents went to recent activity, which the system calls heat. The last five cents went to whether the note was active, waiting, dormant or something else.

That adds up to a whole dollar, and that matters because changing the balance now has an honest price. If importance should get 25 cents instead of 20, those extra five cents have to come from somewhere, usually the close-match share. Nothing appears from thin air. The choice is visible.

A note’s status also became part of the starting score instead of an afterthought. An active or waiting note got the full share for that part. A dormant note got half. Anything else got none. Two notes with the same words could now land in different places for a reason a person can explain: one is still alive in the work, and one is not.

The blank problem

One smaller detail could have spoiled the rewrite. Many notes did not yet have an importance score or a heat score. It would have been easy to treat a blank as zero, and that would punish every note nobody had labeled yet. A missing label is not the same as a bad note.

So a blank got the middle value instead: 50 out of 100, or 0.5 once converted for the formula. In plain language, the system says, “We do not know enough to push this note up or down.” That is much fairer than burying a note because nobody has filled in a box.

What stayed

The time-based bonuses stayed, because they were the part of the old idea worth keeping. Waiting more than five days, being due within seven days, having an overdue follow-up and being updated in the last 48 hours are not permanent facts about a note. They are reasons to move it up the line today. Those bumps still get added after the shared starting score, and the final result is capped at 1.0.

That split made the whole setup easier to defend. Some facts describe what a note usually is: how close it is to the question, how important it is, whether it is active. Those belong in the basic mix. Other facts describe a short-lived squeeze, like a due date closing in. Those get a temporary nudge.

If a system ranks your work, ask how it decides what comes first, and whether closeness of match is quietly outweighing importance and due dates.

dxdev LESSON · paste into your AI coding agent
LESSON: Stacking urgency boosts on top of a 100 percent similarity base score is an unstated weighting decision
SOURCE: dxdev.com/everyday-ai/old-note-that-kept-winning
WHAT HAPPENED: A retrieval system ranking notes for an AI to read used cosine similarity to the question as nearly 100% of the base score, then added four flat bonuses on top for being overdue, due soon, waiting, or recently updated. Because similarity effectively WAS the whole starting score, an old and inactive note that happened to share wording with the question could outrank a genuinely important, active, overdue one, and the target of relevant results more than 80% of the time was not being met. The fix replaced the implicit all-in-similarity base with an explicit weighted split (60% similarity, 20% importance, 15% recency, 5% status) that sums to a fixed total, defaulted a missing importance or recency score to a neutral midpoint rather than zero (so an unlabeled note is not punished as if it were confirmed unimportant), and kept the original time-based urgency boosts as additive nudges applied on top of that explicit base.
THE RULE: When a score's base is implicitly 100% one signal and everything else is a flat bonus added on top, that IS a weighting decision, just an unstated one; making the split explicit and summed to a fixed total is what makes changing it later have a visible, accountable cost.
CHECK MY CODE, then report PASS or FAIL with file:line for each:
  1. Any ranking or scoring formula whose base score is effectively 100% one signal (similarity, recency, a single metric) with other factors layered on as unexplained flat bonuses.
  2. Any missing or null numeric field defaulted to zero inside a score calculation, where zero would treat not yet labeled the same as confirmed unimportant or inactive.
  3. Any multi-factor score whose component weights are not documented and do not sum to a fixed, known total.
THEN PRINT: a table (check, PASS/FAIL, evidence, fix) + a verdict (applies / partially / OUT_OF_SCOPE / no) + the single most important next action.