Release 1.6 of Handee, the compliance system we build for multi-site property operations, changed how one of its own numbers is calculated. The number went down.
Nothing about the underlying work got worse. The old figure had been flattering us since the first release, and this week we stopped it.
A number that could only go up
Handee reports an operator's on-time completion rate, the share of finished jobs that landed on or before the date they were due. Until this week it was measured against the first time a job was marked complete.
There was a reason for that, and it was not a good one. Until this week there was no way to record that a job had come back. A completion was final by construction, so the first one was the only one there was.
The code was doing exactly what it had been told to do. A rule that stops counting at the first success can only ever produce good news, which is why a supplier writes one without noticing.
What the old number was hiding
When a contractor's work fails and somebody has to send them back, the second visit has to be recorded somewhere. With no way to reopen the original job, the only place to put it was a new work order.
So the system held two jobs. Both completed. Both, quite possibly, inside their due dates.
The failure had become an additional success, and the operator's completion figures improved in proportion to how badly the work had been done.
Release 1.6 keeps rework on the original work order, carrying a mandatory reason for why it came back, and scores on-time completion against the final completion rather than the first. It also adds a figure that had nowhere to live before, the share of finished work that was never sent back at all.
Operators will see their on-time rate fall. We would rather explain that in advance than let somebody find a number that dropped overnight and wonder what else moved.
Shipping the denominator
Every number we added in this release ships beside a plain statement of what it cannot see.
The rate for work fixed first time is published with the three blind spots that inflate it, one of which is that a job sent back to a different contractor never appears in it at all. The ratio of reactive to planned maintenance is published with the share of jobs it was able to classify, because work raised before this release carries no classification and we did not invent one for it. Supplier concentration is published with a count of the awards that carry no value, so that unpriced work cannot be read as absent work. A count of status changes with no recorded actor sits on the dashboard as a data-integrity signal, which is a polite way of saying that some of our own history is thinner than we would like it to be.
None of this makes the product look better.
A figure that arrives alongside a list of its own exclusions is a figure somebody has to think about, and a dashboard full of those is harder to sell than one full of large green percentages.
A number without a denominator is decoration that happens to be numeric, and an operations lead who acts on it is making a decision we misled them into. The customer-facing version of this release is on the Handee site.
Why any of this should matter to you
If you are choosing who to hand part of your operation to, every candidate will show you a dashboard, and the dashboards will look much the same. They will all be green.
The useful question is what those numbers were measured against, and what the supplier did on the day the honest answer turned out to be inconvenient. Whether they have ever moved a metric in a direction that cost them something. Whether they will tell you what a figure cannot see before you have relied on it, rather than after.
The number we promise clients is the human time recovered on the work they hand us. A promise like that is only worth something from people who will also tell you when one of their own numbers was wrong.