Mean Reciprocal Rank (MRR)
MRR is the average, across many queries, of 1 divided by the rank of each query's first relevant result — rewarding an early correct hit.
Mean reciprocal rank (MRR) is the average, across a set of queries, of the reciprocal rank of the first relevant result each query returns — 1 divided by that rank — so a system that consistently places the right answer first scores near 1.0, and one that buries it scores close to 0.
MRR is built for a specific shape of task: questions with exactly one right answer to find, like "what is the top document that answers this," rather than tasks where multiple relevant documents matter equally. It rewards getting the first good result close to the top and punishes burying it, harshly and non-linearly.
How it works
For each query, find the rank position of the first relevant result, and take its reciprocal: rank 1 scores 1.0, rank 2 scores 0.5, rank 5 scores 0.2, rank 20 scores 0.05. Average those reciprocal ranks across every query in the evaluation set, and that average is the MRR.
The 1/rank shape is deliberately steep near the top: the jump from rank 1 to rank 2 costs 0.5 of a point, but the jump from rank 10 to rank 11 costs less than 0.01 — meaning MRR is highly sensitive to whether the very best result is right at the top, and nearly indifferent to what happens far down the list. That makes it the right metric when a user only ever looks at the first result (or a system only ever reads the first chunk), and the wrong metric when multiple results matter equally — recall@k is the better fit there.
Why it matters
MRR is a standard scoreboard for question-answering and search-ranking systems specifically because it is single-number, easy to track over time in CI, and directly interpretable: an MRR of 0.5 means, on average, the first relevant result lands around rank 2. It's the metric of choice for evaluating rerankers, since a reranker's entire job is moving the right answer toward rank 1 — exactly what MRR rewards.
A worked example
Five queries are run against a retrieval system, and the first relevant result lands at ranks 1, 3, 2, 1, and 5.
reciprocal ranks: 1/1=1.0, 1/3≈0.333, 1/2=0.5, 1/1=1.0, 1/5=0.2
MRR = (1.0 + 0.333 + 0.5 + 1.0 + 0.2) / 5 = 3.033 / 5 ≈ 0.607
An MRR of about 0.61 says the system, on average, needs to check somewhere between rank 1 and rank 2 to find a relevant result — dragged down mainly by that one query where the right answer sat at rank 5. Fix that single query's ranking and the average moves noticeably; that sensitivity to individual queries is exactly what makes MRR useful for catching regressions a raw pass/fail check would smooth over.
How miatz teaches it
Learners compute MRR by hand on a five-query result table exactly like the one above, then rerun it after deliberately reranking one query's results — watching a single fixed query visibly move the group average — before ever letting an eval script compute it for a hundred queries at once.