← learning.n0tls.com
§ third in the series, following "The Best Wrong Answer"

Digging for hard negatives,
and what it actually changed

The math said a mined hard negative should matter more than a batch full of random ones. Here's the code that finds them, the training-loop change that uses them, a real CUDA crash along the way, and the 12-run sweep that shows it only paid off for one of the three models.

mine_hard_negatives.py train_biencoder.py eval_embed.py
once per base model
mine_hard_negatives.py
corpus + queries → 2 confusable docs per pair
12 runs · 3 models × 2 scales × 2 ratios
train_biencoder.py
pairs (+ negatives) → fine-tuned model
step 3 · unchanged
eval_embed.py
fine-tuned model → hit rate vs. ground truth
01 finding the confusable documents
mine_hard_negatives.py — new file, one run per base model

The mining loop is short on purpose: it's the exact same encode-then-argsort shape as eval_embed.py, run against training queries instead of eval queries. That's not a coincidence — mining is retrieval. The question "what would this model currently return for this query" is the same question whether you're scoring it or about to correct it.

1

Encode the corpus and every training query, once each

→ cosine similarity
1doc_vecs = model.encode(doc_texts, batch_size=32, normalize_embeddings=True)
2query_vecs = model.encode([p["query"] for p in pairs], batch_size=32, normalize_embeddings=True)
3
4sims = query_vecs @ doc_vecs.T  # (1372, 457) cosine similarity

Same brute-force matrix multiply as the eval script — 1,372 training queries against 457 docs is a bigger matrix than eval's 45×457, but still nothing a GPU notices. This line is why mining has to run per base model: model.encode is base-model-specific, so sims is a different 1,372×457 matrix for MiniLM than for bge-small. "Confusable with the right answer" is a fact about a specific model's pretraining, not a fact about the corpus.

2

Take the nearest docs, skip the true one

→ "H", explained in The Best Wrong Answer
1for i, p in enumerate(pairs):
2    order = np.argsort(-sims[i])
3    hard_negatives = []
4    for j in order:
5        candidate = doc_paths[j]
6        if paths_equal(candidate, p["doc"]):
7            continue
8        hard_negatives.append(candidate)
9        if len(hard_negatives) >= args.top_k:
10            break

Walk the ranked list nearest-first, and take the first top_k (2, here) that aren't the pair's own true document. That's the whole mining step. Whatever survives this filter is, by construction, the model's own best guess at "which wrong document looks most like the right one" — literally the H vector from the loss writeup, computed for real instead of assumed.

the risk this doesn't remove: a "hard negative" mined this way might be a real, acceptable answer to its query, not a wrong one. eval/ground_truth.json tracks this distinction for the 45 eval queries via acceptable_docs — but that oracle doesn't exist for the 1,372 synthetic training queries. There's no code fix for this here; some false negatives are baked into every mined pair, same caveat "The Best Wrong Answer" raised before this script existed.
02 giving the loss more to chew on
train_biencoder.py — two new flags, no new concepts

Both changes plug directly into InputExample and MultipleNegativesRankingLoss, which already had the hooks for exactly this — nothing about the training loop itself needed to change.

1

Extra texts on an example become extra negatives

→ MultipleNegativesRankingLoss docs
1texts = [args.query_prefix + p["query"], doc_text(p["doc"])]
2if args.num_hard_negatives > 0:
3    for neg in p.get("hard_negatives", [])[:args.num_hard_negatives]:
4        texts.append(doc_text(neg))
5examples.append(InputExample(texts=texts))

InputExample(texts=[query, positive]) was the whole story before; now it can be [query, positive, hard_neg_1, hard_neg_2]. Nothing downstream needed to change to make this work — MultipleNegativesRankingLoss already treats every text past index 1 as an additional negative, on top of whatever the rest of the batch provides for free. The library was built for this; the mining script is what was missing.

2

The temperature dial, exposed

→ softmax temperature
1loss = losses.MultipleNegativesRankingLoss(model, scale=args.scale)

One word added to one line. scale defaults to 20 in the library, and the entire September checkpoint used that default for all three base models without ever touching it. This sweep is the first time it's actually a variable instead of an assumption.

03 a real crash, and what it actually meant
the 10GB card pushed back — and it changed what the sweep can honestly claim

The first sweep pass crashed here, mid-run, for bge-small and e5-small specifically — never for MiniLM. This is CUDA (the GPU's compute layer) refusing to allocate more memory than the card physically has:

1torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 72.00 MiB.
2GPU 0 has a total capacity of 9.62 GiB of which 140.31 MiB is free.

Two extra hard-negative texts per example roughly doubles how many texts one training batch has to embed at once. MiniLM is 6 transformer layers; bge-small and e5-small are 12 — twice the activation memory per text, for a batch that was already about to carry twice as many texts. batch_size=32 stopped fitting; dropping to 16 still didn't for the two 12-layer models, and it took dropping to 6 before training would run to completion.

why this matters for reading the results table: batch_size is also the in-batch negative count. MiniLM's hard-negative runs went from 31 negatives/example (batch 32, baseline) to 17 (batch 16: 15 in-batch + 2 mined) — a modest cut. bge-small's and e5-small's hard-negative runs went from 31 negatives/example all the way down to 7 (batch 6: 5 in-batch + 2 mined). That's not a clean "mined vs. random negatives" comparison for those two models anymore — it's confounded with "7 negatives vs. 31," a cut forced by VRAM, not chosen as part of the experiment. Only MiniLM's row pair below is close to an apples-to-apples test of hard negatives on their own.
04 what the sweep actually found
12 runs · 3 base models × scale {8, 20} × mined hard negatives {0, 2} · same 45-query eval set · what recall@k and hit@1 mean
modelscalehard negsbatchexact@1acceptable@1recall@5
MiniLM803237.8%51.1%82.5%
MiniLM821637.8%53.3%85.0%
MiniLM2003235.6%44.4%87.5%
MiniLM2021626.7%44.4%80.0%
bge-small803233.3%42.2%77.5%
bge-small82631.1%40.0%77.5%
bge-small2003233.3%42.2%75.0%
bge-small202628.9%40.0%70.0%
e5-small803228.9%40.0%82.5%
e5-small82633.3%42.2%75.0%
e5-small2003235.6%44.4%82.5%
e5-small202631.1%42.2%77.5%

Highlighted rows: best config per model, by recall@5 first, acceptable@1 to break ties. One run per configuration — no fixed random seed, so single-point gaps are noise, not signal.

MiniLM: the one clean win

baseline, recall@5
82.5%
+ 2 hard negs
85.0%
baseline, acceptable@1
51.1%
+ 2 hard negs
53.3%

Both scale-8 rows for MiniLM improved over their in-batch-only baseline, on every metric that moved at all. It's also the model this was hypothesized to help most — the weakest base model of the three, with the most room for a large gradient on a modest amount of signal to actually help.

bge-small and e5-small: hard negatives never won

Every hard-negative row for these two models scores at or below its matching in-batch-only row, on every metric. Read alongside §03's confound, the honest conclusion isn't "hard negatives don't help deeper models" — it's "this sweep couldn't isolate whether they do, because the VRAM-forced batch-size cut buried the signal under a much bigger negative-count cut." deferred a re-run with gradient accumulation, so effective negative count doesn't have to collapse to fit 2 mined negatives in memory, would actually test it.

One number moved independently of hard negatives, though: bge-small's best two rows are both scale 8, never scale 20 — mild, not dramatic, support for the earlier hypothesis that its already-well-calibrated pretrained space wants a gentler dial than MiniLM's does.

Check your understanding— optional, 5 questions

1. Why does mine_hard_negatives.py need to run separately for each base model rather than once for the whole corpus?

2. What real risk does mining hard negatives from nearest-neighbor similarity introduce, one the eval set's acceptable_docs field guards against for eval queries but not for training queries?

3. Why did appending hard-negative texts to InputExample's text list work with zero changes to MultipleNegativesRankingLoss's own code?

4. Why is the hard-negative comparison for bge-small and e5-small called "confounded," while MiniLM's isn't?

5. What pattern in the sweep suggests loss scale (temperature) matters somewhat independently of hard negatives?