Table of Contents

A job usually starts with a gap. One region closes deals at half the rate of another. One site turns away three times as many claims as the rest. One recruiter fills a role in twelve days while another takes thirty.

The instinct is to explain the gap. The better first move is to check whether it is real.

Definitional variance

Across engagements in several sectors, a consistent share of reported performance gaps has turned out to be partly or wholly an artefact of measurement rather than a difference in performance.

The mechanism is mundane. Two groups report the same metric name. They compute it differently. Nobody has compared the definitions because the name matches, and a matching name reads as a settled question.

Concrete versions I have run into:

• Denial rate calculated by one site against claims submitted and by another against claims adjudicated in the period. Different denominators, same label, a gap that partly evaporates on reconciliation.
• Time-to-place starting at requisition approval in one team and at first candidate contact in another, with roughly a week of difference sitting in the gap between those two events.
• Response time measured from ticket creation in one queue and from first human read in another, where an overnight batch import shifted the clock by hours.

None of this involves anyone behaving badly. Each definition was locally reasonable when it was set.

The reconciliation step

Before root-cause work, a short procedure:

1. Get the actual formula from each group. Not the metric name and not the dashboard. The computation: numerator, denominator, time window, exclusions.
2. Ask what is excluded and why. Exclusions are where definitions drift most, and they are rarely documented.
3. Recompute both sides on one definition. Whichever definition you prefer, apply it uniformly, then look at the gap again.
4. Report how much of the gap survived. This number matters and is worth stating plainly.

This takes days, not weeks, and it changes what the engagement is about often enough to justify the time.

Why it is worth insisting on

Two reasons.

The first is that diagnosing a gap that does not exist produces a fix for nothing, and the fix has a cost. Worse, when the metric fails to move afterwards, because there was nothing to move, the conclusion drawn is usually that the intervention was wrong rather than that the problem was misstated.

The second is that the reconciliation itself frequently surfaces the real finding. If two sites have been reporting incomparable numbers to the same leadership meeting for two years, that is a live problem in its own right, and generally a larger one than the gap that prompted the review.

Limitations

This is practitioner observation, not a measured result. I have not run a controlled study of how often reported gaps survive reconciliation, and the organizations where I have seen this were not randomly selected. They were organizations that had already concluded something was wrong, which plausibly biases toward messier measurement than average.

The claim is narrow: the check is cheap, it is skipped by default, and it pays often enough to be worth making routine.