Precision, Recall, and F1 for Match Rules: A Worked Example

Precision is the share of pairs your match rules called a match that are actually the same entity; recall is the share of true matches your rules found. This piece defines and computes precision, recall, and F1 for scoring your own match rules ahead of a go-live decision. It does not cover how to build the underlying labeled truth set, and it does not cover how to source or generate realistic duplicate test data; the entity resolution test data guide linked below owns that question.

Three numbers describe how a set of match rules performed against a labeled truth set, once every candidate pair has been bucketed into true positive, false positive, true negative, or false negative.

Precision is TP divided by (TP plus FP): of the pairs your rules called a match, what share actually are the same entity. Precision answers "when my rules say match, how often are they right."

Recall is TP divided by (TP plus FN): of the pairs that are actually true matches in the truth set, what share your rules found. Recall answers "of the duplicates that exist, how many did my rules catch."

F1 is the harmonic mean of the two: 2 times precision times recall, divided by precision plus recall. F1 answers "how do precision and recall trade off into one number," which is useful for comparing rule versions, and risky to read in isolation, for reasons the next section makes concrete.

Here's the arithmetic, run against an illustrative set of numbers you should replace with your own truth-set counts. Nothing in this example is a measured result from a real implementation.

Assume 200 candidate pairs get reviewed against a labeled truth set. The match rules flag 40 of those pairs as matches.

Of the 40 flagged pairs, the truth set confirms 32 are genuinely the same entity (true positives, TP = 32) and 8 are not (false positives, FP = 8).

The truth set itself contains 45 pairs that are true matches in total, across all 200 candidate pairs, not just the 40 the rules flagged. The rules found 32 of those 45, so they missed 13 (false negatives, FN = 45 - 32 = 13). The remaining 147 pairs are correctly identified non-matches (true negatives, TN = 200 - 40 - 13 = 147), which checks out: 32 + 8 + 13 + 147 = 200.

Precision = TP / (TP + FP) = 32 / 40 = 0.80. Recall = TP / (TP + FN) = 32 / 45, approximately 0.71. F1 = 2 x Precision x Recall / (Precision + Recall) = 2 x 0.80 x 0.71 / (0.80 + 0.71), approximately 0.75.

Every number in this example is illustrative, not a measured customer result. Substitute your own pair count, your own truth set's true-match total, and your own rules' actual flags, and the same three formulas produce your real numbers.

Precision and recall trade off, and the direction of that tradeoff tells you which risk you're carrying into go-live. A rule tuned for high precision and low recall is conservative: when it calls a match, it's usually right, but it silently misses true duplicates rather than risk a bad merge. Those missed duplicates survive go-live as fragmented records under two truth keys.

A rule tuned for high recall and low precision is aggressive: it catches more true duplicates, but it also merges some pairs that aren't actually the same entity, creating bad golden records that then have to be un-merged later.

Neither tradeoff is universally wrong. Which one you can live with depends on the domain and what a bad merge costs against what a missed merge costs. That's a judgment call a gate review should make deliberately, with the numbers in front of it, not one that gets made by default because nobody measured which way a rule leans.

F1 is useful for one thing: comparing two versions of a rule set with a single number. It's a poor thing to report alone at a gate review, because two rules with the same F1 can be leaning in opposite directions, and F1 doesn't tell you which. The formula is symmetric in precision and recall, so a rule at 0.80 precision and 0.71 recall and a hypothetical rule at 0.71 precision and 0.80 recall land on exactly the same F1 while carrying opposite risk profiles: one under-merging, one over-merging.

Reporting precision and recall separately, alongside F1, is what lets a Silent Failure Finder rank which specific pairs the rules got wrong, and lets a Gate-Review Certainty Report state, per rule, which failure mode is actually present, not just how good the blended score looks.

None of this arithmetic works without a labeled truth set to run it against, meaning candidate pairs already tagged match or no-match by a known answer key. Building one by hand is real labor. A Match-Ready Sample, shaped to your domain with known duplicate pairs already tagged to a truth key, gives you a truth set to run your own match rules against and compute your own precision, recall, and F1, using exactly the three formulas above, on your own data instead of this page's illustrative numbers.