Many people are aware of the famous quote from Peter Drucker, “What gets measured, gets managed.”  But what people don’t often consider is that what’s being measured and managed might actually not matter at all at the end of the day.  When we measure things that don’t actually drive us to improve, we’re just acting like a cargo cult — going through the motions and expecting something magical to happen that never will.  Instead of just picking some popular or standard metric, we should instead make sure that we understand what direction our measurements are going to give us, and what change they’re going to drive in our behavior or in our products.  Only by being thoughtful and deliberate in our choices will we find the right metrics to use and rely on in our choices.

Here are a few things to watch out for and to take into account when considering what metrics we want to put into place…

Just Say No to “Vanity Metrics”…

We’ve all seen them – metrics that people want to see simply because they look good on paper or in a PowerPoint presentation.  These metrics are usually easy to collect, easy to describe, and completely useless for making decisions.  Not only are they not actionable, usually due to a lack of context associated with the data, but they’re also easily manipulated — either deliberately or incidentally.  Page views, registered user counts, downloads, and other metrics that we commonly see (and are even more commonly asked for by executives) but which are purely vanity-driven.  Sure, it feels really good to have a thousand new registered user accounts per day — but if none of those people ever come back to your site, is it really a good measure?  Or maybe 5,000 people have directly downloaded  your whitepaper — but where did they get the link and why are they interested?  And if 10,000 users viewed your landing page, were they actual users or some botnet scan directed at your website?

Instead of looking at these one-off, feel-good metrics, we need to pick metrics that have context to them — customer journeys through your site, site usage depth maps, time-on-page, time-per-visit, what checkbox options people are using, average users per month.  All of these metrics have two things in common: they actually tell us something about our users, and they are metrics that we can trace a cause-and-effect relationship to.  We can look at the root cause or a blocking point in the process, and make decisions that will actually move these metrics.  In short, these metrics matter because we can affect them clearly and without too many intervening causes.

Decide on the “Why” Before Considering “What”

Far too many people start tracking metrics and implementing responsive action without first considering why they’re collecting the data in the first place.  Metrics should never exist in a vacuum; they should never be tracked “just because”.  Rather, they should be tracked because they are indicators for some strategic or tactical goal that you want to achieve — your strategy should drive your metrics and not the other way around.  All too often, when we implement metrics without considering the reasons for them, we read them wrong or we make false assumptions about the reasons that they matter, which results in us basically running around like chickens with our heads cut off…without rhyme or reason or any sense at all.  Instead, if we take some time to determine what our overall strategy is, or what our tactical options are, we can craft our metrics in the light of those considerations — we will pick better measures that are more representative of the goals we want to achieve, rather than just being the hot metric of the moment.

Understand Why We Measure Anything At All

We don’t measure things to make us feel good about ourselves.  We don’t measure things to make others feel good about us.  We don’t measure things just because other people do.

We measure things to drive change in our product, our process, and our organization.  End of story.

And in order to know what changes our metrics are telling us to make, we need them to be:

• Comparative – We want our metrics to be comparative because we need to be able to measure and express change over time; rarely, if ever do we have a single point in time at which we take a measure, then ignore it or them from that point forward.  If the goal is improvement, then we need to be able to understand how that measure is changing over time, as we make changes in behavior.
• Understandable – We need out metrics to be understandable because we can’t afford for others to need a 10-minute explanation of them in order to assess their importance or impact.  If the goal is improvement, then everyone needs to clearly understand what the metric is telling us, so that we can discuss our behavior and not the metric or the logic behind it.
• Expressed as a Rate or Ratio – Ideally, our metrics are best expressed as a rate or ratio — average users per month; converted customers per week, etc.  Rates and ratios build into them some of the most important components of the SMART goal framework — they’re clearly measurable and establish for us what the timeliness of the goal should be.

Only when a metric hits all (or at least the first two) of these classifications is it really worth our time, and even then only if we’ve considered what its effect on our behavior is going to be.  We can track every single metric ever thought of, but if we’re not using them to change our behavior in some meaningful way, then they’re all useless to us..

Ultimately, Be Realistic

Having said all that, i’ll completely admit that we are quite often asked to produce, deliver, and study metrics that meet none of these elements.  Executives will clamor for them; board members will require them in decks; investors will balk if they don’t see them discussed.  This is the unfortunate state of the world in which we live in — and we’re usually not exactly in a place to be able to tell our CEO, “That’s not a useful metric, so I’m not going to collect it for you.”  So, the reality is, you’ll have to collect these metrics; and you’ll have to report them.  But you should also be sure that these vanity metrics aren’t the only ones that you’re collecting; you should ensure that you’re also collecting the kinds of metrics that will drive your behavior, that will lead to improvements that directly push your strategy forward, and that meet all of these descriptions above.  Sometimes we must do the dirty deed in order to free ourselves to also do the right thing — and metrics is often one of these situations.