Cohort Retention Analysis

The Retention Analysis feature allows you to accurately measure product retention and stickiness over time. Product retention measures the percent of users within a cohort that interact with your API in some way and then return continue to interact with your API. Unlike subscription retention, which merely tracks whether a customer has an active subscription, product retention tracks whether users are coming back actively use your product.

Note that events may take up to 20 minutes to show up in the user analytics screens. If your data has not shown up, please try again later once the data has had time to filter over.

If you’re not familiar with retention analysis, you should first read Chapter 3 of API Analytics: The Ultimate Guide to Grow Your Platform Business eBook

Retention intro

In order to start a retention analysis, you need to decide on initial and returning criteria. By default, any activity is considered, but you can specific criteria based on API fields or custom actions.

For example, if you are building a payments API, you can set the initial API call to a customer’s first POST /payments and then the returning call a second POST /payments.

Moesif Retention Analysis

Determining First Event and Returning Events

Creating a retention analysis requires you to define two events that define when a customer is considered retained. The initial event to define is known as the First Event and represents when a user first receives value from the product. For example, on an e-commerce platform, this could be the first time the user actually creates a purchase order. If you are a communications platform, the first event may a user sending a text message through the platform.

The second event you’ll need to define is when a user performs a Returning Event, signaling that they are a repeat customer. The returning event is when a customer returns back to the platform and performs an event where they receive value from the product. Typically, this is the same type of event that was performed in the First Event, but it could be something completely different. Every time a user performs the Returning Event they are considered a retained user. When a user no longer performs the Returning Event, the user is no longer retained and likely is no longer using the product.

Setting the Event Criteria

These filters set up the criteria for the first action and returning action. The results chart is broken down by cohort date in the vertical direction (date of first action). For each row which is a single cohort of users, you can see the percentage of users performing the returning action for each interval after the cohort date.

In the picture, we are showing daily retention, which means day 1 signifies one day after the cohort date, day 2 is two days after the cohort date, and so on.

If instead you selected weekly or monthly retention interval, then a user will be counted for week 1 if they performed the returning action any day of the following week after the cohort date.

For example, if you’re showing daily retention, then your user base is segmented into daily cohorts based on when they performed the first API call.

Configuring Segmentation

By using a Segmentation clause we can make the retention analysis only include retention for customers who performed a certain action or belong to a certain segment of our customer base. This can help users see how specific segments are performing in terms of retention instead of grouping all retention together and creating an average, which would be the case without segmentation. For example, we may add a segmentation clause where we want to look at the differences in retention for both Enterprise and Non Enterprise customers.

Moesif retention chart with segmentation

Configuring Group By

Another angle to look at the analysis from is to use the Group By capability. This allows us to see what the retention looks like across different cohorts of users based on demographic information or acquisition channels. Under the Group By heading on the retention analysis you can add a Group By criteria to group the customers based on a the selected field. For example, we may want to group users by UTM Source. This will show us what customer retention looks like for users depending on what UTM source they came from. Here’s what that would look like.

Moesif retention chart with segmentation and group by

Setting the retention interval

You can set the retention interval by selecting a value from the Retention Interval dropdown.

retention interval selector

Available intervals include:

  • daily
  • monthly
  • weekly

Setting the retention analysis time span

You can set the retention analysis time span by selecting the interval from the dropdown. You can use one of the pre-created date ranges (such as Last Hour, Last 24 Hours, etc) or use an absolute date range based on a calendar date.

Time span selector

All Users vs New Users only

You can also define which users should be included in the retention analysis.

new and all users button

New Users will filter each cohort by only new users. New Users is helpful to measure how sticky your adoption funnel is while excluding users who are already long term customers (hence already likely to have high retention rates).

Alternatively, you can also select All Users to include all users in the analysis. This is the default when a new analysis is created.

Inspect profiles

inspect profiles button

By enabling Inspect Profiles, in the table below the chart you will be able to see users that have recently entered into the retention analysis by expanding the tables and clicking on the number in the Total Users column.

total users column in retention table

A modal will appear with the User ID’s of users who completed the First Event on that day.

modal showing user profiles in retention analysis

Creating a simple Retention Analysis

Creating a Retention Report in Moesif is easy to do. The first step is to navigate to the Users screen. Once on the Users screen, pick Retention from the dropdown in the top left of the screen.

In app navigation to the Moesif retention screen

Now on the Retention screen, you can specify both the First Event and the Returning Event. In the example below we create a simple retention analysis where the First Event and Returning Event will be when a user makes a purchase and a repeat purchase on the platform. We will also make sure to only include successful checkouts by specifying that the response Status Code is equal to 200 or 201, denoting a successful call.

Basic retention chart in Moesif

In Moesif, if First and Returning events are not specified, Moesif will calculate the retention based on any event being received for that user. This can sometimes be useful but is generally more accurate to pick specific First and Returning events. As you can see, Moesif automatically creates a chart that plots out the customer retention curve.

Interpreting the results

When actually looking at a retention report, it’s important to know what you are looking at. There are a few scenarios that show healthy retention and others that show unhealthy retention or retention that could use improvement.

Retention is high/optimal

The optimal retention rate varies by industry so it’s hard to say exactly where you want your retention rate to land. Depending on your industry, you obviously want to make sure that your retention curve begins to flatten quickly and stays roughly around the target for your business/industry. It’s also important to take a look at what you are doing to have retention be so high and continue to do so to keep loyal customers retained. If retention starts to drop as you make changes to onboarding, business processes, marketing, and other factors, you may want to look at reverting back to tactics when retention was higher. Likely the experience of your service or app is not meeting the customer expectation. Your customer retention strategy should always aim to keep retention as high as possible.

Retention drops off quickly

If retention drops quickly to low levels, this usually means that customers are not satisfied with the product or are not getting the value out of the product to justify returning. If you are looking at retention on a monthly level and by the second month retention is dropping significantly, you may want to look at retention in a smaller time segment, such as weekly or even daily. With poor retention, the silver lining is that small changes may lead to massive differences.

The retention report doesn’t show much history

If your business has only been around for 4 weeks or you’ve only been tracking retention for 4 weeks, it’s tough to look at retention on a 12-month basis since the data doesn’t exist. If you find that your retention report stops sooner than you’d expect, you’ll need to make sure that you have enough data to support the time span that you are trying to look at. If your data is limited, consider using a smaller unit of time such as looking at retention on a daily or weekly basis until your data grows big enough to assess at a monthly or yearly level.