A hard negative is a document that looks right and isn't — and it turns out to matter more to a contrastive loss than almost anything else in the training set. Here's the math for why, and what it implies for tuning three different base models instead of one.
The last checkpoint left four things deliberately unbuilt. This picks up the second: hard-negative mining and per-model hyperparameter tuning, held off on last time rather than tuned blind. The recipe that produced the September checkpoint — 1,372 synthetic pairs, one LLM pass, in-batch negatives only, one hyperparameter setting shared across three different base models — helped MiniLM on every metric and hurt bge-small on every metric except recall@5. Before changing anything, it's worth understanding why a training recipe can point in opposite directions for two models doing the same job.
The short version: almost none of the 1,372 pairs' in-batch negatives were doing any work. The gradient a contrastive loss produces is concentrated almost entirely on the few negatives that are close enough to the right answer to be confusable — and how close "close enough" is depends on a hyperparameter that was never tuned per model. The rest of this page derives that claim from the loss function itself.
A bi-encoder turns a query and a document into two vectors, q and d, in the same space — 384 dimensions for the small models in this experiment. Both are normalized to length 1, so they live on the surface of a hypersphere rather than filling the space around it. That normalization is what makes a single number, cosine similarity, stand in for "how related are these two":
That single scalar is the entire interface between "what the model believes" and "what the loss function can push on." Everything below is about what happens to that number, and its neighbors, during one training step.
Set the equations aside for a second. Every training step is really a multiple-choice question: the model is handed a query, the one true document, and 31 other documents that happened to land in the same batch of 32. It already has a similarity score for each of the 32 — the cosine number from section 1. The only question left is what the model does with those 32 numbers.
A rule called softmax turns the 32 raw scores into 32 percentages that add up to 100% — read each one as "how confident the model is that this particular document is the right one." A higher score gets a bigger share, and the gap gets stretched on the way there: a candidate whose score is only a little higher than another's can end up with a lot more of the confidence, not just a little more.
Run fig. 1's four candidates through this rule and the shape of the problem is already visible:
The loss itself is almost an afterthought once step one is done. It's just: how far was the confidence on the correct answer from 100%?
One dial sits inside step one, before any of this: a temperature, written τ (sentence-transformers exposes its inverse as scale, defaulting to 20). It controls how aggressively softmax stretches small score gaps into large confidence gaps. Fig. 2 already used the library's default; section 4 comes back to what changes when that dial turns.
Here's the one piece of calculus worth sitting with, because everything else in this page follows from it. Training doesn't mark each candidate "right" or "wrong" — it nudges every candidate's score by a specific amount, and that amount is just the gap between the confidence it got and the confidence it should have gotten (100% for the true document, 0% for everything else):
Plug fig. 2's numbers in and the nudges are: true document, 0.803 − 1 = −0.197 (pulled up, since the gradient is negative); mined hard negative, 0.197 − 0 = 0.197 (pushed down); moderate negative, 0.0003 (pushed down, but by almost nothing); easy negative, a number with eight zeros after the decimal (pushed down by an amount too small to matter). Two candidates get moved. Two don't.
Look again at the true document's own nudge, 0.197 in fig. 3. That number only exists because H was in the batch, competitive enough to take 19.7% of the confidence for itself. Drop H and keep only M and E, and the true document's confidence would sit above 99.9% — the model already "won" that comparison trivially, so its own nudge would shrink to almost nothing too. A hard negative isn't just teaching the model to push one distractor away. It's what stops the true document's own gradient from disappearing. Without one, a batch full of easy negatives can leave a whole training step doing close to nothing — even for pairs the model already gets right.
The 1,372-pair recipe never chose its negatives — each query's negatives were whatever else happened to land in the same 32-example batch, drawn from a 457-document corpus with real topic clustering (installation docs next to other installation docs, API references next to other API references). Most of the time, a random draw from a 457-document corpus lands on something topically unrelated to the query: an E-type easy negative from fig. 2, contributing a nudge too small to matter. Batch size 32 gives 31 candidate negatives per query, but per the math above, it's not the count that matters — it's whether any of the 31 happen to be confusable, like fig. 2 and fig. 3's H. On a corpus this size, with random sampling, that's a matter of luck per batch rather than a designed property of the training set.
That's the case for mining negatives rather than accepting whatever the batch hands over: deliberately search for documents that are close enough to be worth a gradient, instead of waiting for one to show up by chance.
eval_embed.py already produces.This is the mechanism H represents concretely, back in fig. 1 through fig. 3: not a random draw, but specifically the runner-up the model was most tempted by. It's also why mining needs a trained-enough model to start from — mining against an untrained model's arbitrary embedding space would surface arbitrary neighbors, not meaningfully confusable ones. The base checkpoints (MiniLM, bge-small, e5-small) are exactly good enough for this: they weren't trained on this corpus, but they weren't trained on nothing, either.
A "hard negative" mined this way might not be a negative at all — the eval set already distinguishes an ideal_doc from a broader list of acceptable_docs for exactly this reason: some queries have more than one legitimately correct document. A mined neighbor that happens to be a real acceptable answer, labeled as a negative and trained to be pushed away, actively teaches the model something false. Any mining pass needs to check candidates against known acceptable-document lists before treating them as negatives, and where that list doesn't cover a corpus's full structure, some false negatives should be expected rather than assumed away.
τ (or its inverse, scale) is the dial from section 2, and turning it changes fig. 2's percentages without touching a single similarity score. High scale exaggerates small score gaps into large confidence gaps — that's what produced fig. 2's 80.3% / 19.7% split at the library's default of 20, even with a genuinely close competitor at 0.90. Turn the dial down to a gentler scale of 8, and the same four scores land very differently:
Low scale spreads confidence out across more of the field; high scale hands nearly all of it to whichever candidate is currently winning. Neither is free: fig. 4's gentler setting gives more of the field a voice, but it also gives more voice to noise if a "moderate" candidate only looks related by coincidence.
Nothing about the correct scale is universal — it depends on how similarity scores are distributed in a particular base model's embedding space before training even starts, and that distribution is a property of how each model was pretrained, not something the fine-tuning recipe controls. bge-small was the strongest base model of the three, and the only one that beat every agent baseline off the shelf. A model whose pretraining already spread its cosine similarities out sensibly, across a range that generally tracks relevance, doesn't need aggressive sharpening — and pushing it hard against noisy, single-pass synthetic pairs is one plausible explanation for why fine-tuning made bge-small worse on every exact-match metric while it made MiniLM, the weakest base model, better on all of them. A model with more room to improve has more to gain from a large gradient on a modest amount of possibly-noisy training signal; a model that already had this figured out has more to lose.
That's a hypothesis, not a finding — the last checkpoint deliberately didn't test it. It does turn "why did one recipe help one model and hurt another" from a mystery into a specific, checkable claim: look at scale and hard-negative ratio per model, not one setting shared across all three.
Concretely, three things get added to what train_biencoder.py and eval_embed.py already do:
acceptable_docs list, and attach a small number of hard negatives to each training pair rather than leaving negatives to whatever lands in the batch.None of that is built yet. This page is the part that had to be understood before touching the pair-generation or training scripts — the next step is writing the mining pass itself.
1. Why does normalizing q and d to unit length let cosine similarity be computed as a plain dot product?
The dot product of two vectors equals ‖q‖‖d‖cos θ in general. Once both vectors have length 1, the norms are both 1 and drop out, leaving exactly cos θ — that's the whole reason fig. 1's math works.
2. In fig. 2/3, why do the "moderate" and "easy" negatives get almost no gradient nudge?
The nudge on a candidate is its confidence minus its target (0 for any negative). A candidate softmax barely notices gets a confidence near 0, so its nudge is near 0 too — it just doesn't move.
3. What's the practical case for mining hard negatives instead of waiting for a random batch to include a confusable document?
With real topic clustering in a corpus, most random negatives are easy negatives (E-type) — mining deliberately searches for the confusable ones (H-type) that actually move the loss, instead of hoping one shows up by chance.
4. Turning the temperature/scale dial down (e.g. 20 → 8) does what to the softmax distribution?
Fig. 4: at scale 8 the mined hard negative's share nearly doubles and the moderate negative finally gets a nonzero nudge. The dial redistributes confidence among candidates already in contention — it doesn't manufacture contention out of nothing.
5. Why might a well-calibrated pretrained model like bge-small benefit less from aggressive hard-negative mining or a high scale than a weaker model like MiniLM?
This is the hypothesis section 4 raises: a model with more room to improve has more to gain from a large gradient on noisy signal; a model that's already well-calibrated has more to lose from the same aggressive push.