Gate-Review Certainty Report: What Good Looks Like

Last updated 2026-08-06.

Definitional boundary. This page defines the report and its scorecard row structure. It does not cover how to compute the precision, recall, and F1 that fill the scorecard, or how to build the labeled truth set the rules are scored against in the first place; those are separate pages.

A gate-review report has five parts, and leaving one out turns evidence back into an assertion with a cover page. First, a scope and domain statement names which domain, which match rules, and which source systems were in scope for this scoring pass, so nobody downstream has to guess what "validated" covers. Second, a per-rule scorecard, the Go-Live Precision Scorecard, states precision, recall, and F1 for each rule individually rather than one blended number for the whole rule set. Third, a ranked failure list, the Silent Failure Finder, lists the specific failures the scoring surfaced, ordered by how much each one matters to the domain, with the fix applied or recommended next to it. Fourth, a re-test delta, the Post-Tune Re-Validation Pass, shows the same scorecard run again after tuning, so the reader sees the before and the after side by side instead of taking "we fixed it" on faith. Fifth, a sign-off line names who reviewed the report and when. A report with a precision number and none of the other four is a status update wearing a report's clothes.

Each of the five earns its place by answering a question the room will actually ask. The scope statement answers "what exactly did you check," which matters because "we validated match rules" means something different for one domain than for three. The per-rule scorecard answers "which rules are actually the weak ones," which a single blended figure cannot answer at all. The failure list answers "what did you find," and a scorecard without one leaves the room to assume the answer is "nothing," whether or not that is true. The re-test delta answers "did the fix work," and without it the room has to take that on faith the same way it took the original number on faith. The sign-off line answers "who is accountable for this document," which is the question that turns a PDF into something a delivery lead can actually stand behind at the meeting.

The Go-Live Precision Scorecard is a table with one row per match rule, not one row per domain. Collapsing every rule into a single domain-wide average is exactly what hides the failure a steering committee needs to see. Each row carries eight columns: Rule ID, the field or fields the rule matches on, match type (exact, fuzzy, or phonetic), precision, recall, F1, known failures found, and status.

That row shows the shape a scorecard takes, not a result: every value in it is a placeholder, not a measurement from any engagement. The point is a structure a reader can build their own scorecard against. Eight columns, one row per rule, nothing averaged away inside a single blended number.

Each column earns its place. Rule ID exists because a failure list is useless if nobody can tell which rule it came from. Field(s) matched exists because a rule scoring well on name alone is a different claim than a rule scoring well on name plus address; conflating the two hides exactly the distinction a steering committee needs. Match type exists because an exact-match rule and a fuzzy-match rule fail differently and need different fixes, even when their precision numbers look similar. Precision, recall, and F1 sit as three separate columns rather than one blended figure, for the same reason F1 alone hides a rule that is high-precision but low-recall behind a rule that is balanced but mediocre at both: the room needs to see which failure mode a given rule actually has. Known failures found ties the scorecard to the failure list rather than leaving the two documents to drift apart. Status turns the row into a decision, not just a measurement: pass, tune, or hold, stated plainly enough that a reader who has never seen the underlying data can still follow what happens next.

An F1-only summary with no per-rule breakdown is theater. So is a report with a precision number and no failure list: it tells the reader the rules were scored, not what was wrong with them or what changed after tuning. A report someone else can re-derive from, meaning another analyst could take the same labeled truth set and the same match rules and reproduce the same scorecard, is not theater. That reproducibility is the actual bar, not the topline number. If the only way to trust the report is to trust the person who wrote it, it is not a certainty report yet, whatever it is titled.

Theater tends to look competent from a distance, which is what makes it dangerous at gate review. A single slide with a precision number and a green checkmark reads as confident. It also cannot answer the follow-up question a careful reviewer asks next: which rules, which fields, which failure mode. A report built row by row from a scorecard can answer that follow-up without anyone opening a spreadsheet mid-meeting, because the answer is already sitting in the document, not locked in whoever built it.

The report is not a one-off deliverable dropped at the end of a project. It sits at the end of a sequence: score the rules against the labeled truth set, then tune the rules that failed, then report the result. That order, score before tune before report, is not arbitrary: the report's re-test delta only makes sense in that sequence, since there is no before-and-after to show without having tuned in between. Treating the report as a step in an ongoing process, rather than a document produced once at the finish line, is what keeps the scorecard current the next time a rule changes.

That framing also changes what "done" means for the person writing it. A report treated as a finish-line document gets written once and filed. A report treated as a stage in the score-tune-report sequence gets rerun the next time a rule is edited, a new source system comes online, or a threshold moves, because the sequence itself does not stop after go-live; it repeats every time something changes underneath the rules it is measuring. Building the report with that repetition in mind, rather than as a one-time artifact, is what keeps the next version of it cheap to produce instead of a re-do from scratch.