Dynamic Sampling

Dynamic sampling is a feature available on enterprise plans which enables large cost reduction by dynamically sampling API traffic while ensuring data accuracy even with elaborate rules around sample rates.

Companies that have a limited budget for analytics, yet have high volume (millions to billions of API calls per day) will benefit drastically from the cost reduction. Sampling requires no code change and can be changed on the fly.

Impact of sampling

Sampling enables your plan’s event quota go further because events that were not logged due to sampling do not count against your quota. For example, if your API usually sees 1 billion API calls per month, but you set the global sample rate to 10%, you only need a plan for 100 million events per month. Because Moesif has full knowledge of your sampling rules, the platform is intelligent enough to extrapolate what the true metrics should be. This means you’ll still have accurate reporting regardless of defined sample rates.

How to set sample rates

Moesif supports three levels of sampling:

  1. User/Company level sampling. A sample rate specified for a specific user or company (When both set, the user’s sample rate takes priority)
  2. Sample rates for behavioral cohorts. A sample rate defined for a cohort of users or companies such as existing customers who signed up over 90 days ago
  3. Global sample rate

Sample rates are prioritized in that order. Meaning a sample rate set for a single user takes priority over the sample rate for a behavioral cohort that a customer belongs to. Unsurprisingly, the global sample rate has the lowest priority and is by default set to 100% for all customers.

1. User/Company level sampling

Any user or company can have a defined sample rate from 0% to 100%. By default, all users sample 100% of API traffic (i.e. all traffic). Reducing any customer to 0% effectively suppresses all data collection for that user (A mechanism to enable GDPR/CCPA compliance). A use case of leveraging user level sampling is to set your 10 largest customers to a less than 100% sample rate, will collect 100% of API traffic for all others.

To set the sample rate for an individual user or company, click on Users or Companies in the top header, then look up the user or company of interest. Once you open up their profile view, you will see a button Edit Sample Rate.

Customer level sample rate

2. Sample rates for behavioral cohorts

Behavioral cohorts is where the full power of dynamic sampling can be leveraged. It enables you to define any number of sample rates based on customer behavior. For example, you can have one sample rate defined for New companies who signed up in the last 12 weeks which you might want at 100%, but then have a different sample rate for All existing enterprise customers who send more than 1,000 API calls per hour such as 25%. Even with a large number of saved cohorts with defined sample rates, Moesif will still extrapolate the original metrics.

Membership of behavioral cohorts are continuously being evaluated by Moesif enabling users and companies to enter and leave the cohort based on their behavior or demographics.

To set a sample rate for a cohort of users or companies:

First, save a behavioral cohort

This can be done by going to the user or company look up from the secondary header bar. After adding the filters you want for your behavioral cohort, click the _Create Cohort_button at top left and give it a name.

In this example, we are selecting companies who were both created in the last 12 weeks and also performed at least 1 API call to either /widgets/selling/status or /purchases in the last 12 weeks.

Create Company Cohort

Then, set the sample rate for the cohort

Once you have created a cohort, view the list of cohorts under Saved Company[or User] Cohorts from the top header. You’ll see the name of the cohort you created. From there, click the pencil icon to edit the sample rate.

Create Company Cohort

3. Global sample rate

The global sample rate is used by any user or company that doesn’t have its own sample rate nor belongs to any behavioral cohort. To modify the global sample rate, go to Runtime Rules under the top right menu.

Global Sample Rate

Auditing effective sample rate

As shown above under Runtime rules, you can see the effective sample rate for each user and company id along with the global sample rate. This is helpful when a user or company belongs to multiple behavioral cohorts creating ambiguity in the effective sample rate.

Keep in mind, a user’s sample rate always takes priority over the company’s sample rate. Any user or company id not listed under Runtime Rules will default to the global sample rate.