Entity Resolution Test Data: Sample Records With a Ground-Truth Answer Key
Entity resolution test data needs two things the standard benchmark repositories omit: an inline schema a QA engineer can read without loading a paper, and an answer key that states, per record pair, whether the verdict is match or no match and why. This guide provides a labeled sample with its answer key, the evaluation arithmetic to score it, and a generator to extend it to your schema.
What entity resolution test data needs
Every labeled corpus needs two columns a reviewer can read without opening a paper: a canonical identity ID (here, Master ID) and a variant-class label (Duplicate Type). Without that ground truth, "accuracy" claims collapse into summary statistics. The ER-Evaluation user guide notes that without reference data only summary statistics can be monitored (retrieval date 2026-08-01).
A labeled sample: schema and answer key
All rows sharing a Master ID represent the same synthetic identity. Duplicate Type labels how the row varies: MASTER, EXACT, TYPO_EDIT_DISTANCE, PHONETIC_SOUNDEX, TOKEN_REORDER, NICKNAME_ALIAS, SHARED_ACCOUNT, or NO_MATCH. The table itself is the answer key.
| record_id | master_id | duplicate_type | full_name | phone | address | |
|---|---|---|---|---|---|---|
| 1 | 101 | MASTER | Amara Whitfield | [email protected] | (512) 555-0142 | 48 Elm Street, Austin, TX |
| 2 | 101 | EXACT | Amara Whitfield | [email protected] | (512) 555-0142 | 48 Elm Street, Austin, TX |
| 3 | 101 | TYPO_EDIT_DISTANCE | Amera Whitfield | [email protected] | (512) 555-0142 | 48 Elm Street, Austin, TX |
| 4 | 101 | PHONETIC_SOUNDEX | Amara Whitfeld | [email protected] | (512) 555-0142 | 48 Elm Street, Austin, TX |
| 5 | 101 | TOKEN_REORDER | Whitfield Amara | [email protected] | (512) 555-0142 | 48 Elm Street, Austin, TX |
| 6 | 101 | NICKNAME_ALIAS | Amy Whitfield | [email protected] | (512) 555-0142 | 48 Elm Street, Austin, TX |
| 7 | 101 | SHARED_ACCOUNT | Jordan Whitfield | [email protected] | (512) 555-0199 | 48 Elm Street, Austin, TX |
| 8 | 202 | MASTER | Priya Natarajan | [email protected] | (303) 555-0177 | 220 Cedar Ave, Denver, CO |
| 9 | 202 | EXACT | Priya Natarajan | [email protected] | (303) 555-0177 | 220 Cedar Ave, Denver, CO |
| 10 | 303 | NO_MATCH | Amara Whitfield | [email protected] | (720) 555-0101 | 9 Pine Court, Boulder, CO |
Field set: what a realistic ER corpus covers
Evaluation arithmetic: TP, FP, TN, FN, pairwise F1
Bucket predictions into the four cells below, then compute Precision = TP / (TP + FP), Recall = TP / (TP + FN), and F1 = 2 × Precision × Recall / (Precision + Recall). The ar5iv practitioner guide notes pairwise F1 and cluster-level metrics can disagree on the same dataset, so track both for clustering systems (retrieval date 2026-08-01).
| Bucket | Meaning |
|---|---|
| True positive (TP) | Correctly matched pair |
| False positive (FP) | Incorrectly matched pair |
| True negative (TN) | Correctly separated pair |
| False negative (FN) | Missed match |
The ER-Evaluation user guide states around 400 resolved entities is generally sufficient for accurate performance estimates (snippet only; page was fetch-blocked; verify before treating as settled fact).
Incumbent map: where to find established benchmarks
| Benchmark | Size | Format | Notable limitation |
|---|---|---|---|
| Leipzig benchmarks | Varies by task | Two source CSVs + perfect-mapping CSV | Paper citation required for methodology |
| CompERBench | 21 tasks | Fixed train/test splits | ML-matcher focused |
| BPID (EMNLP 2024) | 20,000 | matching_dataset.jsonl | Synthetic personal identity only |
| RLdata500 / RLdata10000 | 500 or 10,000 | R package | German name/age sources; R-only |
| pseudopeople | Simulated US population | Python package | Requires install; simulant_id ground truth |
For algorithm depth and a downloadable name-only sample, see the fuzzy matching guide for entity resolution. For a full-customer-record corpus with Master ID labels, use the customer records dataset for resolution testing.
Generate a larger version
exact_matching and/or fuzzy_matching in the free generator, then export CSV or JSON with Master ID and Duplicate Type columns. Anonymous use is free with no signup; sign in for larger exports and the full format menu.Frequently asked questions
What is a ground truth set in entity resolution?
A ground truth set is a collection of record pairs annotated as match or no match by a human reviewer. It is the reference you measure your matching system against. The AWS Entity Resolution post defines it as a small subset of pairs a human has annotated and notes it need only be large enough to be representative (retrieval date 2026-08-01).
How do you calculate pairwise F1 for entity matching?
Bucket predictions into true positives, false positives, true negatives, and false negatives. Precision = TP / (TP + FP). Recall = TP / (TP + FN). F1 = 2 × Precision × Recall / (Precision + Recall). Pairwise F1 and cluster-level metrics can disagree on the same dataset, so both are worth tracking for clustering-based systems (ar5iv practitioner guide, retrieval date 2026-08-01).
How much labeled data do I need to evaluate an entity resolution system?
The ER-Evaluation user guide states that benchmark datasets of around 400 resolved entities are generally sufficient for accurate performance estimates (snippet only; page was fetch-blocked; verify directly). The AWS post notes the set need only be large enough to be representative of the real match distribution (retrieval date 2026-08-01).
Can I test entity resolution without real customer data?
Yes. BPID (EMNLP 2024) is a 20,000-record synthetic personal identity deduplication benchmark with no real PII. The pseudopeople package simulates a fictional US population with a simulant_id ground truth column. The labeled corpus on this page is also entirely synthetic (AWS Entity Resolution blog, retrieval date 2026-08-01).
Is record linkage the same as entity resolution?
The terms refer to the same task. Wikipedia lists data matching, data linkage, and entity resolution as synonyms for identifying records that refer to the same real-world entity (en.wikipedia.org/wiki/Record_linkage, retrieval date 2026-08-01).
What is the difference between deterministic and probabilistic record linkage?
Deterministic linkage applies explicit rules: if fields match exactly (or after normalization), records are linked. Probabilistic linkage assigns a match score based on how likely the observed field agreement pattern is under a match versus non-match model (Wikipedia record linkage, retrieval date 2026-08-01).