The truest indicator of whether your product is providing real value to the users is retention. Very simply, it’s a metric that tells whether your users are coming back to your product regularly or not.

How to measure retention

Here are four steps that you should follow:

1. Pick a leading indicator of revenue and repeat behavior: If you’re a marketplace with two sides, you need to have metrics for both the supply and the demand side. For example, for Airbnb: Rebook rate (% of customers that rebook after the first booking) and number of nights booked per user over time are good examples of demand side metrics/actions that truly indicate that the user is using the product. A vanity metric like app downloads or pageviews here doesn’t indicate much. On the supply side, metrics like number of bookings per unique active host over time is a good supply side metric.
2. Pick the right time period for your company: Depending on the business you’re in, think about how frequently you want your ideal user to use your product. In case of a product like Airbnb, you can imagine since people don’t travel often, the focus should be on measuring retention on an annual basis, not weekly or even monthly. For a product like Commonlounge, our focus would be to measure retention on a weekly basis since we expect users to come back once a week (not daily, not monthly).
3. Identify an initial user action within the first period: 100% of the users being tracked for retention purposes take this action, and it’s usually a leading indicator of revenue. For Airbnb, this is booking a room for at least one night for the first year. For Commonlounge, this would be subscribing to a list on the first visit.
4. Identify a follow on user action in the subsequent periods: For Airbnb, it could be looking at subsequent years after the sign up, and seeing if a night was booked in Year 2, 3, 4 and so on. In case of Commonlounge, it could be a completion of a tutorial in Week 2, 3, 4 and so on. Calculate the % of install base that is still engaging in that action in period 2, 3, 4 and so on.

A set of users with a same property is called a cohort. So all the users who sign up in a particular week (or year, if yearly retention makes sense for you) is called that week’s (or year’s) cohort. So for Airbnb, since they have been in business for a few years, if they were to plot out their retention curve, it will look something like this:

Note that each of these like represent a set (or cohort) of users that signed up in a given year. Here’s the most important thing: after a few periods (let’s call it the Long Term Period), your retention curve must stabilize (or even go up, like Airbnb’s Yr 1 curve goes up after 3 year mark) close to the average long-term retention targets for your business vertical. Here are a few verticals, and their typical Long Term Targets for retention:

Source: How to Set Up, Staff and Scale a Growth Program

This is the gist of measuring retention for startups. Once you have this nailed down, you can start looking into building out your growth team and scaling your product.

Account expansion and reactivations

In addition to your retention curve flatlining, it’s a very good sign if you see account expansion — that is, your existing users spend more time/money (or whatever your goal is) in your product. In addition, it’s also great if users get re-activated — that is, users who used your product in the past and didn’t use it for a while come back and start using it again. There are three main ways this can happen:

1. Network Effects start kicking in: If your product is something that gets better as more and more people join in (think social networks like Facebook, or even marketplaces like Airbnb and Uber — more the hosts/drivers, the better your experience as an end user), you will see your retention curve go up over time as your product becomes more valuable for everyone.
2. Product become available on a new platform: Think of a social network that launches its mobile app — so people can now get push notifications instead of emails for new activity, and hence end up spending more time on the product (We saw this happen first-hand on Commonlounge)
3. Product expands into new categories: Think of Amazon expanding from just books to other categories. Suddenly, the site become a lot more useful for the same person, seeing more repeat usage. In context of Commonlounge, this would mean creating lists on new topics.

Magical moments

This topic is a little fuzzy, but we the broad gist is that as a product person, it’s important to remember the power of magic moments. These are the moments that serve as peaks in your user’s experience, and are usually correlated with real value being provided by your product to them. These are also the moments that make them prod their friends and tell them about your product, and why you’re cool. Thinking of your user’s product journey in terms of magical moments, and intentionally creating them is a good thing to keep in mind.

Why is retention so important?

Retention is a truer indicator of if you’re doing something right than any other metric. If you look at just an aggregate metric like total minutes spent, or total monthly revenue, it’s not a true indicator of the health of the business.

Why? Because of the Forest fire theory of startups. Imagine a forest fire starts in the middle of the forest, and it spreads outward. At any point of time you look, you’ll see more trees burning as the fire spreads outward, and you’d be happy to see all the aggregate metrics charts up and to the right.

In the illustration above, green are the people who haven’t tried out your product yet (the forest), maroon are the people who are trying out your product at any given moment (the fire), and black is for people who tried your product but decided not to use again (burnt forest)

Over time though, since your retention is ~0, we know that the fire will consume the entire forest and eventually die. This is what happens with a lot of products that are not able to get to a stable line of retention — some of them may even seem to be highly successful while the fire is spreading (since the red area is increasing over time). A core reason why this happens is that the product inherently doesn’t provide real value to anyone.

Conclusion

This was a brief overview of retention, and how to measure it. A lot of tools like Heap Analytics give you great, out-of-the-box tools for plotting these retention charts — it’s very easy to get started and this is the first and the most important metric you should measure.