Proxy Measures and Their Half-Life
When the real thing cannot be measured, something correlated is used instead. All proxies decay, and knowing how fast matters.
A proxy stands in for something you cannot measure directly. It works because of a correlation, and the correlation is the thing that erodes.
The measurement warning in “Proxy Measures and Their Half-Life” matters whenever software records work patterns. Organisations researching remote employee productivity monitoring can use explore Monitask for time and project context, while outcomes, quality checks and direct feedback remain necessary to explain what the metric cannot show.
Common proxies
Documents produced, standing in for analysis done.
For an independent perspective related to “Proxy Measures and Their Half-Life”, consult the NIST Privacy Framework; it offers a useful external check on definitions, governance and the assumptions built into a proposed measure.
Meetings attended, standing in for involvement.
Code committed, standing in for software delivered.
Responses sent, standing in for support given.
Each is correlated with the real thing at the moment it is chosen.
Why they decay
The correlation existed under the behaviour that preceded measurement.
Once measured, behaviour shifts toward the proxy, which is the Goodhart note.
And the organisation changes: new tooling, new processes, new work mix, all of which weaken the original relationship.
The half-life idea
Assume any proxy is substantially less informative after a year and close to useless after two.
Not as a rule of arithmetic but as a planning assumption.
It means a proxy needs a review date from the day it is adopted, which almost none has.
Testing whether it still holds
Periodically, compare the proxy against the real thing on a sample.
Take twenty items, assess them properly by judgement, and see whether the proxy ranks them similarly.
An afternoon, annually, and it is the only way to know whether the measure is still measuring.
When it has broken
Retire it.
This is harder than adopting it, because the figure is in reports and somebody built a target around it.
Its own note covers retiring a measure, and the practice is the difference between a measurement system and an accumulation of legacy numbers.
Using proxies honestly
Label them as proxies in the reporting.
"Documents produced, as a proxy for analytical output" is a different claim from "analytical output".
Readers adjust their confidence accordingly, which is the whole point of saying it.
The multiple-proxy approach
Several weak proxies that would each be gamed differently are harder to game together.
And where they diverge, the divergence is informative.
This is more robust than searching for one good proxy, which usually does not exist.
What to check
Which of your measures are proxies rather than direct measures?
Are they labelled as such?
When was any of them last tested against the real thing?
And does each have a review date?