Dynamic Sampling

Dynamic sampling is a cost-savings feature available on enterprise plans which gives you control on how much data is logged to Moesif based on customer and API behavior. This can be done on the fly with a few clicks in Moesif. No code change or restart is needed.

Because these API calls are not logged to Moesif, they do not count against your subscription’s event quota. As long as Moesif sees a portion of API traffic, Moesif automatically extrapolates original metrics so your reporting is accurate within an error of margin.

You can create sampling rules based on customer demographics, user behaviors, and also on specific regex matches on API fields like URI or status code.

How Moesif works with dynamic sampling

Dynamic Sampling is available only on enterprise plans. Companies with high-volume APIs (millions or billions of API calls per day) stand the most to benefit from the cost savings from dynamic sampling.

How sampling works

With sampling Moesif only sees a portion of your API traffic based on behavioral rules you define which can be user behavior, their demographics, or other criteria. Because Moesif doesn’t see all traffic, you can save cost since you can use Moesif on a higher-volume API beyond your plan as sampled API calls do not count against your quota. For example, if your API usually sees 1 billion API calls per month, but you set the global sample rate to 10%, you only need a plan for 100 million events per month. Because Moesif has full knowledge of your sampling rules, the platform is intelligent enough to extrapolate what the true metrics should be. This means you’ll still have accurate reporting regardless of defined sample rates.

Dynamic Sampling works only on API Calls. Actions triggered from a UI are not sampled.

How to set sample rates

Moesif supports four levels of sampling (prioritized in this order):

  1. A static rate for a single user or company
  2. Sample based on user behaviors and demographics (saved cohorts)
  3. Sample based on regex rules on API fields like certain URIs
  4. Global sample rate

Sample rates are prioritized in the above order. If both the user and company has a defined sample rate, the user’s rate always takes priority over the company’s rate. Meaning a sample rate set for a specific customer takes priority over the sample rate associated with a behavioral cohort that a customer belongs to. Unsurprisingly, the global sample rate has the lowest priority and is by default set to 100% for all customers.

To access dynamic sampling, go to Alerting & Governance within the the top header menu and then select Sampling & Suppression

Overview of sampling and suppression

User behavior-driven sampling

Behaviorally driven sampling is the most powerful and common way to leverage dynamic sampling. With behaviorally driven sampling, you apply a sample rate to members of a behavioral cohort which can be something like “Companies who made over 1M API Calls last 4 weeks” or “Newly integrated users who made purchase transactions”. You can create both a company rule or a user rule.

For high reliability, set your global sampling rate to your “typical” sampling expected for most customers such as 50% or 100%. Do not set global sampling to 0%. Then, use behavioral cohorts to adjust that sample rate higher or lower for specific customers. Behavioral cohorts are dynamically refreshed every few minutes. However, there is a small window where a user or company that was just enrolled into a cohort may not have their changes propagated to your server integration yet, causing the user/company to fall back to global and regex-based sampling rules.

To set a sample rate for a cohort of users or companies:

1. Save a behavioral cohort

If you have not created a behavioral cohort (which drives sampling rules, governance rules, among other things), you will need to create one. This can be done by going to the user (or company) look up from the header bar. Add the customer demographics or behavioral filters you want for the cohort. Once done, click the _Create Cohort_button at top left and give it a name.

In this example, we are selecting users who were created in the last 4 months and performed at least 1 API call to either /widgets/buying or /purchases in the last year.

Create User Cohort

2. Set the sample rate for the cohort

Once you have created a cohort, go back to Sampling & Suppression under Alerting & Governance. Click + User Rule or click + Company Rule based on what on what you created in step 1. From here, you will see a list of saved behavioral cohorts. Select one and then set the sample rate.

Apply Sample Rate to User Cohort

In this case, 100% means log all API calls, whereas 0% means suppress all data collection.

Static rate for a single user or company

Finally, any user or company can have a defined sample rate from 0% to 100%. By default, all users sample 100% of API traffic (i.e. all traffic). Reducing any customer to 0% effectively suppresses all data collection for that user (A mechanism to enable GDPR/CCPA compliance). A use case of leveraging user level sampling is to set your 10 largest customers to a less than 100% sample rate, will collect 100% of API traffic for all others.

To set the sample rate for an individual user or company, click on Users or Companies in the top header, then look up the user or company of interest. Once you open up their profile view, click the orange button Edit Sample Rate.

Regex rules

With Regex rules, you can dynamically adjust sample rates based on specific fields within the API call. For example, you may want to log any response status code > 400 at 100%. You can also use regex rules to suppress data collection on uninteresting traffic such as GET /health/probe. API Calls do not need to be attributed to a user or company for regex rules to work which make them perfect for health probes and test traffic.

To get started, click the orange + Regex Rule. This will open a panel where you can select the fields and the regular expression to match.

Add Regex Rule for Dynamic Sampling

Select the field to match on such as the Request Route or the Response Status. Then, add your regular expression. If you need help, view regex101 to build one.

Then select the sample rate that should be applied for this regular expression. In the above example, we want to suppress data collection for non-interesting traffic by not logging health probes.

Global sampling

The global sample rate is used by any user (or company) that doesn’t have its own sample rate nor belongs to any behavioral cohort. To modify the global sample rate, click on “Global Sampling” button

Global Sample Rate

This also brings up a few other options to suppress traffic such as on IP address or known bots and crawlers.

Auditing effective sample rates

There are two ways to audit the effective sample rate for a customer. The effective rate include both any statically defined sample rate for a user/company such as for GDPR compliance and also any sample rates dynamically set by behavioral cohort rules. It does not include non-customer centric rates like regex rules which are applied on a per-request level. To view effective sample rate for a customer:

  1. Going to any user or company profile and view the Effective Sample Rate section.

  2. Select Dynamic Sampling from under the Alerts & Governance header bar. Then click the orange Settings button at the top right. This will open a pop up like below:

Effective Sample Rates

The time when your cohort rules were last updated is also displayed. This means the most recent time when either you manually saved the cohort criteria or because a user/company was added or removed from a cohort based on their behavior. If the cohort hasn’t been updated, this could imply no customers have been added or removed recently.