The first version of my context-pack ranker scored every node as cosineSimilarity + a pile of additive boosts. It looked reasonable. It was quietly broken in a way that no test would catch, because the math ran fine and the numbers came out in range. The bug was conceptual. By making similarity the whole baseline and bolting everything else on as bonuses, I had decided, without ever deciding it, that cosine similarity was the only thing that mattered.

This is the retrieval ranker for a note-management app I’m building to hand an AI a relevant “context pack” for whatever you’re asking about. The pipeline is the usual shape. Embed the query with OpenAI text-embedding-3-small (1536-dim), cosine-compare against stored node embeddings, rank, take the top. The whole V0 success metric is blunt and unforgiving: context packs are relevant 80%+ of the time, measured by daily use. If they’re not, I refine retrieval and I do not get to go build UI instead. So the ranker is load-bearing, and getting the scoring formula wrong means the whole product feels dumb.

The old formula, and why “plus bonuses” is a trap

Here’s what V0 did, in spirit:

finalScore = baseSimilarity + boost

where boost accreted on top:

  • +0.30 if the node had been waiting more than 5 days
  • +0.25 if something was due within 7 days
  • +0.20 if a follow-up was overdue
  • +0.15 if it was recently updated

Read that and ask one question: where do importance, heat, and lifecycle state enter the score? They don’t. They aren’t in the base, so the only way they could ever influence ranking was as another ad-hoc bonus. Similarity carries its full weight of 1.0, untouched, and everything I actually care about is a footnote stapled to the end.

That’s the trap with similarity + bonuses. It feels modular and additive and tunable, but it isn’t. You haven’t built a scoring system with knobs. You’ve built “cosine similarity, lightly nudged,” and every nudge is a special case you wrote because the pure-similarity ranking embarrassed you on some specific query. Each bonus is a patch over the fact that your base only knows one thing. The bonuses don’t compose into a model. They just pile up.

The concrete failure mode: a node that’s a perfect semantic match but dormant, abandoned, nobody-cares-anymore, will outrank a slightly-less-similar node that is genuinely active and important right now. Because similarity is 1.0 of the base and “active and important” was never given a seat at the table. You can keep adding bonuses to claw that back, but you’re fighting your own baseline.

The rewrite: fold the steady-state signals into a weighted base

The fix was to stop treating similarity as the baseline and start treating it as one weighted input among several. New base:

score = (baseSimilarity * 0.6)
      + (importance     * 0.2)
      + (heat           * 0.15)
      + (stateBoost     * 0.05)

Similarity is still the heaviest term at 60%, which is correct, semantic relevance should dominate. But importance now gets 20%, heat (a recency/activity proxy) gets 15%, and lifecycle state gets 5%. These are real weights. They sum to 1.0. If I decide importance should matter more, I move 0.2 to 0.25 and pull it from similarity, and I can reason about exactly what that trade costs me. That is what “tunable” actually means: a dial where moving it does a predictable thing, not a growing list of if statements each canceling out a different past mistake.

stateBoost is worth calling out because it turns lifecycle into a first-class ranking input instead of an afterthought:

stateBoost = 1.0  for active or waiting
           = 0.5  for dormant
           = 0.0  otherwise

So an active node and a dormant node, all else equal, now separate in the base score, by design, before any time-pressure logic runs. In the old model the only way “this thing is dead” could affect ranking was if I happened to write a bonus for it. Now it’s structural.

The null-handling detail that quietly matters most

Here’s the bit I’d have gotten wrong if I weren’t paying attention:

importance ?? 50   // then / 100
heat       ?? 50   // then / 100

An un-scored node defaults to 50, which after the /100 is 0.5, the neutral midpoint. Not zero.

That default looks like a rounding decision, and it is the whole ballgame. Most of my nodes aren’t annotated with importance or heat yet, because I haven’t gotten around to it. If importance ?? 0, then every un-annotated node eats a zero in a term worth 20% of the base, and the ranker silently buries everything I haven’t manually scored. The system would punish me for not having finished labeling my own data, and the failure would be invisible, just slightly-worse rankings I’d never trace back to a ?? 0.

Defaulting to the neutral midpoint says the honest thing: “I don’t know how important this is, so don’t let that opinion move the score either direction.” Un-scored is neutral, not worthless. When you fold a signal into a weighted base, the default for a missing value is the midpoint of its range, never zero, unless you genuinely mean “absent equals bad.”

What stays additive, and why

I didn’t delete the bonuses. The four temporal-urgency adds still stack on top of the weighted base:

score += 0.30  // waiting > 5 days
score += 0.25  // due within 7 days
score += 0.20  // follow-up overdue
score += 0.15  // updated within 48h


score = Math.min(1.0, score)

then clamp to 1.0 so nothing runs away.

The distinction I landed on, and the thing I’d reach for again next time: steady-state relevance should be a blend, time-pressure should be a bump.

Similarity, importance, heat, state, those describe what a node is. How relevant, how important, how active, what lifecycle stage. Those are properties that coexist, so they belong in a weighted blend where they trade off against each other in fixed proportions. “Waiting more than 5 days” and “due within 7 days” are different in kind. They describe a transient condition the node is in right now, a reason it should jump the line today specifically and not tomorrow. That’s a genuine bump on top of its steady-state score, and modeling it as additive is correct precisely because it’s temporary and out-of-band.

The old formula had only one place to put any signal, the bonus pile, which is how importance ended up homeless. Once you separate the two kinds, you get a clean answer to the question every ranker eventually has to defend: why did a slightly-less-similar node outrank a perfect match? Because it was more important, currently active, and overdue, and the formula can show its work on each of those.

Stack: tRPC, Drizzle, MySQL on Railway, GPT-4o for parsing, text-embedding-3-small for vectors.