How to choose the right Lean UX metrics
Deciding which Lean UX metrics matter to you is fundamental for growth and success, here’s how to choose the right ones
- Chapter 1 – How to Apply the Lean UX Methodology
- Chapter 2 – 4 ways companies can scale a Lean UX process
- Chapter 3 – Lean UX vs Lean Startup: Are they the same thing?
- Chapter 4 – Lean UX principles for large companies
- Chapter 5 – How to choose the right Lean UX Metrics
- Chapter 6 – 5 Essential Lean UX tools: A curated list
- Chapter 7 – 5 Lean design hacks to improve product success
The build-measure-learn loop is how Lean UX functions. You build something, see how well it performs and you learn from that. Rinse and repeat. UX metrics are how you measure whether you’re successfully navigating that loop.
Metrics are there to help you understand what’s going on with your product and hopefully offer insight into how to build a better one. Without looking at metrics, you’re flying blind.
Since measurement is a big thing within Lean UX (and pretty much all UX), it helps to know what to measure and how to measure it.
So how do you choose the right Lean UX metrics? And what metrics should you be looking at when building products in a Lean environment? Justinmind will show you how.
Create user experiences that delight. Download Justinmind free.
What are UX metrics?
UX metrics are data that we collect about a range of different variables relating to our products.
Metrics are often coupled with analytics, which when used together can help you make better product decisions. How does that work?
A metric is a piece of information. This information can be anything from the number of users that your product has, what your conversion rate is or even how much money your product is making.
This information on its own isn’t powerful. Knowing how many users you have doesn’t really help you in any meaningful way.
Analytics is when you gather the data and extract sense out of it. Basically, metrics are the data and analytics is the information.
Metrics are used in a variety of ways but, importantly, they help to make sure our vision isn’t a delusion. This is vital when you hope to launch a successful product to market.
By defining and outlining what success looks like early, you’ll be able to understand whether your metrics are good or bad.
If you attempt to build a product without using metrics then either your product will immediately fail or you won’t know how to get better or what to improve, which will result in a slow journey to failure anyway.
Good UX metrics and bad UX metrics
Not all metrics are born equal. It may take some time to convince your key stakeholders that having a large number of likes on your Facebook page is virtually meaningless. Let’s explain.
A good metric is one which is:
What does this mean exactly? Allow us to furnish you with an example:
Let’s say you have a landing page that’s got a 5% conversion rate. Is that a good landing page or a bad one? Right now, we have no idea. All we know is it converts at 5%.
Now, we just mentioned that a good metric should be comparative. If we say that landing pages in this sector generate a standard 3% conversion rate, what do we start to learn?
Because we are able to compare against a similar product, the metric starts to make more sense. We’re able to see if things are improving (or not) by comparing metrics. In this example, our landing page that converts at 5% is stellar.
Good metrics also need to be actionable. If we know that the standard conversion rate in our industry for a landing page is 3% but ours is at a meager 1%, we can start to figure out why and take the steps to improve it.
Is the copy persuasive enough? Is the price too high? Are we solving our users’ problems? These are just a few questions which you can put to the test to get to the bottom of your problem. The metric must be actionable because as UX designers we need to take action to change user behavior.
Understandable and measurable are the last two things that make a good metric. If a metric isn’t understandable or measurable then your team will be unable to make any decisions about how to improve the product you’re building.
A measurable metric could be something like the growth of active users over a period of time. See how this metric is more meaningful than the more general metric of how many users you have?
What is a bad metric?
A metric is a bad metric when it doesn’t help you make a decision.
Vanity metrics, for example, are very easy to collect. They make us feel good or bad about our product. But, unfortunately, they don’t do much more than that. The number of likes on a tweet or on a Facebook status update doesn’t tell us anything about the user experience of those services.
Another vanity metric might be the number of downloads your product has. Let’s say it has a million downloads. Aside from making our ego blow up, does the metric tell us anything else?
Not really, it just tells us the number of people who have downloaded an app. It doesn’t say anything about their experience with the app, whether they used the app, deleted it or even recommended it.
When it comes to Lean UX, a bad metric would be one that takes up a lot of time and resources, especially if they don’t provide comparatively as much value. Any metric that doesn’t really provide any information of value is typically bad.
Having too many metrics can be considered bad. Metrics exist to help us make better product decisions and if you’re drowning in many metrics this can result in distraction. It’s better to focus on the right metric.
Qualitative and Quantitative measurement methods
User research can help you understand which UX metrics to focus on. By understanding your users better, you can build products that they want to use. How do we do that? Using qualitative and quantitative research.
Both teach you different things about your user and the impact of your product changes. Put simply, quantitative research is what is going on with your product and qualitative is why that’s going on with your product.
Quantitative research data offers statistically significant data about a product’s user experience. To get this, there are many methods which can be used:
- A/B testing and multivariate
- Card sorting
- Surveys or questionnaires
- Desirability studies
Qualitative on the other hands usually involves being with the user. It helps you understand why something is happening, usually through observation.
Qualitative research is often used when you don’t have a lot of information about the topic you are researching, allowing a level of exploration which is perfect for choosing your Lean UX metric or your One Metric That Matters, which we’ll touch on shortly.
Qualitative methods include:
- User testing
- Talking to users
Generally speaking, you only need 5 users in a qualitative study to uncover 85% of usability problems.
Choosing One Metric That Matters
In Lean there is the concept of the One Metric That Matters (OMTM). Because founders of companies typically have their fingers in all the proverbial pies, the OMTM provides a little focus for the distracted founder. It’s essentially the one number you care about.
Metrics aren’t set in stone. Lean is about adapting to change and if today your OMTM is X but next month it’s Y, no sweat. OMTM is about the most important metric in your current stage. It all depends where you are in the Lean UX loop.
Defining your one metric that matters should be done early. The earlier the better. As your product grows, you’ll invariably have more metrics to look at but when you’re a small team it pays to get your head around the one metric that matters.
As mentioned, you need to know what success is going to look like. In the Lean Analytics book, there’s a case study about Airbnb. The product team had a hypothesis that better photos would boost rentals.
In true Lean fashion, Airbnb created an MVP with minimum effort. Their MVP showed that professionally photographed listings received two to three times more bookings than the market average. They continued to build on this success.
They experimented further by adding watermarks to photos for authenticity, customer service offered professional photographs to renters and they increased the requirements on photo quality.
Their metric was to track how many shoots were taking place per month because they had already validated that professionally photographed listings received more bookings. In the end, Airbnb was doing 5,000 shoots a month.
The benefits of One Metric That Matters
In their book Lean Analytics: Use Data to build a Better Startup Faster, Benjamin Yoskovitz and Alistair Croll outline 4 main reasons for using the One Metric That Matters:
- It answers the most important questions you have
There’s a lot that goes on in a startup. Many questions, many answers. By identifying the one key metric that is important to your success as a company, you’ll maintain focus in spite of persistent distraction.
- It forces you to draw a line in the sand
By defining OMTM, you’re drawing a line in the sand and giving yourself clear goals. Pick a number, set the target and have confidence that you’ll hit it – success. If you don’t, simply make adjustments and try again.
- It focuses the whole company
OMTM can be used to focus your entire company. Croll and Yoskovitz suggest displaying your metric throughout the office, on TV screens, in email and on web dashboards. By giving your OMTM pride of place, your team will always know what the company is trying to achieve.
- It inspires a culture of experimentation
Experimentation is fundamental to Lean. To move through the build-measure-learn cycle quickly and frequently, you have to experiment.
Of course, there will be failures (many of them, actually) but planned failure will move everything forward and ever closer to success. As it states in Lean Analytics: “when everyone rallies around the OMTM and is given the opportunity to experiment independently to improve it, it’s a powerful force.”
What makes the OMTM interesting is that it highlights where to move your focus next. By focusing all your efforts on one metric, you’ll start to see which metric needs attention. Like we said, your OMTM isn’t set in stone and is likely to change.
If your OMTM was to drive traffic to your website and you achieve that your next step would be to optimize that traffic into conversion.
How to choose the right Lean UX metrics – conclusion
No one can tell you which metric is important to you or your business. But, what is vital is that you pick metrics that are important to you right now. That means choosing metrics which are actionable, measurable, comparable and understandable – stay away from the temptations of vanity metrics, they won’t help you reach success any sooner.
Over time your metrics can definitely change – such is the Lean UX way – but maintaining focus and repeating the Lean UX cycle is what will drive your product forward.
Continue reading the Complete Guide to Lean UX below