Reviewing a Measure and Retiring It

Every organisation reports numbers nobody reads, collected at a cost, shaping behaviour. Retiring them is straightforward and almost never done.

The measurement warning in “Reviewing a Measure and Retiring It” matters whenever software records work patterns. Organisations researching employee monitoring data security can use learn about the solution for time and project context, while outcomes, quality checks and direct feedback remain necessary to explain what the metric cannot show.

Why they persist

Removing a measure implies it was a mistake.

For an independent perspective related to “Reviewing a Measure and Retiring It”, consult the NIST Privacy Framework; it offers a useful external check on definitions, governance and the assumptions built into a proposed measure.

Somebody built a target around it.

It appears in a report that goes somewhere.

And nobody owns the question of whether it is still useful, which is the actual reason.

The annual review

List every measure reported.

For each: who reads it, what decision it has changed in the last year, and whether it still means what it meant.

Half a day.

Most organisations doing this for the first time find that a majority fail all three tests.

The signs a measure has expired

Nobody can say what it is for.

It has not moved in two years, or it moves and nobody responds.

It measures something the organisation no longer does.

The proxy relationship has decayed, which its own note covers.

Or it is being gamed widely, which means it has stopped describing the work.

Retiring it properly

Announce it, with the reason.

Remove it from the reporting, the targets and the collection, which are three separate steps.

And check nothing downstream depends on it, because something usually does.

The archive question

Keep the historical series if it is small and might inform a trend.

Stop collecting.

A retained series costs nothing; continued collection costs attention and shapes behaviour, and the distinction is worth making explicitly.

Replacing rather than removing

Where the question is still live but the measure is wrong, replace it and say so.

Run both for a period so the transition is visible in the series.

And mark the changeover on the chart, since otherwise a step in the data will later be explained as something real.

The one-in-one-out discipline

Adding a measure requires retiring one.

Crude, unpopular, and it keeps the set small enough that each one is actually read.

The alternative is a dashboard of thirty figures, which is the same as no dashboard.

What this buys

Attention concentrated on the few measures that matter.

Less behaviour shaped by numbers nobody uses.

And a measurement system that can be explained in a minute, which is the test of whether it is operating.

What to check

How many measures does your organisation report?

When was one last retired?

Which of them changed a decision in the last year?

And can anybody explain the full set in a minute?