The Devil’s In The Metrics

Your workforce will be shaped by your metrics. I’ve heard this many times and encountered it plenty. Recently, I’ve learned to see a new side of that problem. Within my own business, a whole host of issues seem to be caused by this phenomena… or rather, how I’ve allowed management to apply it.

Before I get into it, let’s get everyone on the same page. A workforce has to be measured somehow, right? A KPI or “key performance indicator” is supposed to measure whether or not the job is done correctly. The thing is… when a KPI is measuring the wrong thing all sorts of chaos can happen. Classic example: a cashier’s KPI might be the number of items they ring per hour. If management evaluates cashiers based mainly on how many items they ring per hour… customer service, accuracy, and all sorts of other important things can fly out the window.

In my case, I’ve built a piece of software over the last 5 years that’s suited for operating a mail-order repair shop or service center, while also including modern warehouse management features. My business uses it to offer various services to online retailers. It collects a lot of statistics about the workforce which can be very granular. For instance, it’s easy to see exactly which stock units Joe Schmoe touched on Tuesday, when, why, and what the customer was charged for that.

It has this terrible feature… the ability to see, in real-time, how much revenue each user is generating. That feature isn’t intended to be a KPI. It’s meant to help with things like catching incorrect charges, easily answering questions about invoices, and adjusting prices. The problem is, it’s too easy for management to misunderstand or misapply.

Everybody understands dollars and cents

Everybody understands dollars and cents, so when faced with a dashboard of statistics, that’s what they gravitate towards. Mind you… these revenue statistics are not intended as a primary gauge of staff performance. They’re supposed to spot billing and pricing mistakes. However, it’s too easy for a lazy manager to say “Joe… you only generated $60 today, you need to get your numbers up”.

This misuse has contributed to a lot of problems. Certain members of management use the revenue stats as their main gauge for performance and harp on the staff about it. That’s created an environment where many staff think nothing else matters. It encourages staff to game the statistic to get the management off their back, which in turn reduces overall quality.

The software has plenty of quality control features and stats to measure whether or not the work is being done correctly and on time. Those features mean actual QC has to be done. For instance, a supervisor must physically spot check outbound packages and input the data for the relevant metric to populate, and mustn’t cut corners for the metric to be meaningful. The process is straightforward, but doing it doesn’t directly generate any charges to customers. The process is skipped or corners are cut, quality suffers, and management is puzzled…

The puzzle is easy: The focus on individual revenue generated is the wrong metric, and fixating on it causes more problems than it solves.

Even large companies have this issue

Amazon’s alleged issue about employee’s peeing in bottles strikes home for me. A previous staff member told me he wasn’t permitted to use the bathroom or take breaks. No manager actually said that to him. Rather, his primary job wasn’t supposed to generate much in the way of direct charges to customers, but a manager kept chasing him about his “billings”. He told me in order to do his actual job and keep the manager happy he had to skip breaks and avoid using the bathroom… which made both his health and quality of work suffer.

He pointed out the toxicity of this culture and it’s worship of “the one true metric” early on. At the end of the day, revenue does matter, but holding each and every person to the same metric, regardless of job function, only leads to disaster. To me, that was common sense, and I couldn’t accept the idea our managers didn’t understand that. At the time, I honestly didn’t believe him.

As an aside… let this be a lesson in the usefulness of exit interviews and keeping an open mind. That staff member left the company for unrelated reasons, but that allowed him to give the brutally honest feedback we should have payed closer attention to at the time. Common sense isn’t common, and looking back I never took the time to properly explain to managers why this statistic shouldn’t be used in that way. I assumed it was common sense.

Lessons learned?

To those reading this, you might be asking why I let this happen or why I don’t terminate the management who are doing this. The simple answer: I’m still trying to figure that out and how to fix it. The people involved are otherwise pretty good at what they do… but then again, this behavior may outweigh anything else they’re doing. What’s the point of management that pushes staff to do the wrong things?

I see the whole issue as my fault for a couple reasons.

First of all, the individual revenue metric, while useful in certain cases, is too easy to abuse or misunderstand. I assumed it was obvious, and I was wrong about that. I was lazy and didn’t take the time to properly explain it to the management or make sure they’re using it correctly. While I’ve intervened in specific cases, that’s actually backfired because the workforce hear the CEO saying one thing while the management enforces something different.

It may also be a UI/UX issue with the software. It’s too easy to use the metric the wrong way. The old UX axiom of “make it easy to use correctly, hard to use incorrectly” probably applies here. With the intended use being investigating pricing and billing issues, perhaps the stats should only be accessible through some kind of formal “customer claim”. Just some sort of added barrier so management can’t look at them unless they’re investigating a billing mistake or pricing error. I’m not sure what the best approach is.

Now that the toxic misapplication of this metric has taken root it’ll be much harder to fix than if I’d addressed it earlier.

I’m posting this so other people, especially founders and management, can learn from these mistakes. If you’re running a company and it seems like there’s a widespread issue among your staff: look at your metrics and how they’re being applied. In fact, review that constantly. It’s true: the workforce you create will be the one that you measure for. If you measure for the wrong thing, you’ll get a bad fit even if you have great people.