n0tls

Learning

Deep-dive writeups from an ongoing retrieval-research project — why an LLM agent misses the right doc, how far you get without vectors, and whether a small fine-tuned retriever beats it. Each one ends with an optional multiple-choice quiz on the core concepts.

The Retrieval Pipeline, Line by Line
Three files that turn bi-encoder architecture diagrams into something that actually runs: synthetic training pairs, contrastive training, and brute-force cosine search — with the real code for each.
bi-encoders · sentence-transformers · code walkthrough
The Best Wrong Answer
A hard negative is a document that looks right and isn't — and it matters more to a contrastive loss than almost anything else in the batch. The math for why, worked through step by step.
contrastive loss · softmax temperature · hard negatives
Digging for Hard Negatives
The code that mines real hard negatives per base model, a real CUDA out-of-memory crash mid-sweep, and a 12-run sweep showing it only paid off for one of three models.
hard-negative mining · hyperparameter sweep · CUDA