What the number leaves out

Suppose an office gets its average call time down from ten minutes to six. These are made-up numbers, but you can probably picture the meeting. The chart slopes in the right direction. The change gets a green arrow. Something appears to be working.

I would want to know whether people got their problems solved.

Maybe they did. Maybe the staff finally got a tool that puts the right information in front of them, and four minutes of everyone's day stopped being wasted. That would be worth celebrating.

Or maybe the callers learned that there wasn't much point in staying on the line.

The number can't tell those stories apart by itself. We have to go looking for something else.

I like numbers. They can interrupt a story we have become too comfortable telling. A person can insist everything is fine while the records show the same problem happening every week. A good measurement makes that harder to ignore. It can give someone with very little influence a way to say, “Look. This keeps happening.”

But we choose what to count. We choose when the clock starts and when it stops. We decide whether a case marked closed means the work is finished, even if someone else still has to live with it.

There is a whole view of the world tucked into those decisions.

Imagine a person standing at a counter with the wrong document. The employee explains what is missing and moves to the next person. From one side of the counter, the interaction is complete. From the other, it has produced another trip, another request for time off, and another attempt to work out what was supposed to happen the first time.

The employee may have done exactly what the job requires. They may be helping a long line of people with very little room to change the process. I don't learn much by picking a villain before I understand the arrangement.

I do learn something by asking where the unfinished work went.

That is the part I want to notice in software, in an office, or in any system that has learned to report on itself. A task can disappear from our queue because it was solved. It can also disappear because we made it someone else's task. The dashboard may look equally good either way.

The person receiving that work probably notices the difference.

None of this gives us the option of measuring everything. We can't. People need a manageable way to see what is happening, and a report that tries to contain every part of a human experience would be unusable. We will always have to leave things out.

I think we owe a little curiosity to whatever we left out.

When a number improves, I want to hear from someone whose experience should have improved with it. When the story and the number disagree, I would rather spend some time in that disagreement than hurry to protect either one.

There may be a very good explanation. There may be a problem we haven't learned to count yet.

Either way, I wouldn't put the green arrow in charge of ending the conversation.