Dynamic Sampling

Dynamic sampling is a cost-savings feature which enables you to control which API calls are logged to Moesif based on customer or API behavior. This can be done within the Moesif UI and doesn’t require any code changes or restarts. Moesif will push new rules to your server integrations automatically.

Because skipped API calls are never received by Moesif, they do not count against your subscription’s event quota. In addition, Moesif automatically extrapolates and normalizes metrics so your reporting is accurate even if different customers and behaviors have different sample rates.

Example use cases

Below are some common examples how you can leverage dynamic sampling:

  • Sample customers who signed up in the last 90 days at 100% for maximum support visibility, everyone else is sampled at 50%
  • Downsample your largest volume customers to 25% so they don’t consume your quota.
  • Suppress collection of health probes and other low-value traffic.
  • Opt-out specific customers from data collection based on their contract terms and privacy requirements.
  • Ensure all 4xx or 5xx errors are always logged (100% sample rate)

You can create sampling rules based on customer demographics, user behaviors, and also on specific regex matches on API fields like URI or status code.

Overview of sampling and suppression

Dynamic Sampling is available only on enterprise plans. Companies with high-volume APIs (millions or billions of API calls per day) stand the most to benefit from the cost savings from dynamic sampling.

How sampling works

With sampling, Moesif only sees a portion of your API traffic based on behaviors which can be any of:

  1. User or company behavior (such as whether they signed up in the last 90 days or they experienced an error in the last 7 days).
  2. User or company demographics (such as free vs paid customers).
  3. API attributes such as status code or URI.

Because Moesif doesn’t see all traffic, you can reduce your Moesif subscription cost as sampled API calls do not count against your quota. For example, if your API usually sees 1 billion API calls per month, but you set the global sample rate to 25%, you only need a plan for 250 million events per month. Because Moesif has full knowledge of your sampling rules, the platform is intelligent enough to extrapolate usage metrics even if different customers have different sample rates.

Dynamic Sampling works only on API Calls. Actions triggered from a UI are not sampled.

How to set sample rates

Moesif supports four levels of sampling (prioritized in this order):

  1. Sample API calls for a single user or company
  2. Sample API calls based on regex rules on API fields like specific URIs
  3. Sample API calls based on user behaviors and demographics (via saved cohorts)
  4. Sample API calls globally

Sample rates are prioritized in the above order. If both the user and company has a defined sample rate, the user’s rate always takes priority over the company’s rate. Meaning a sample rate set for a specific customer takes priority over the sample rate associated with a behavioral cohort that a customer belongs to. Unsurprisingly, the global sample rate has the lowest priority and is by default set to 100% for all customers.

To access dynamic sampling, click your avatar at the bottom left, then select the Dynamic Sampling menu option.

Dynamic Sampling Menu

User behavior-driven sampling

Behaviorally driven sampling is the most powerful and common way to leverage dynamic sampling. With behaviorally driven sampling, you apply a sample rate to members of a behavioral cohort which can be something like “Companies who made over 1M API Calls last 4 weeks” or “Newly integrated users who made purchase transactions”. You can create both a company rule or a user rule.

For high reliability, set your global sampling rate to your “expected” sampling you want for most customers and do not set global sampling to a very low number like 1%. Then, use behavioral cohorts to opt-in groups of users/companies to higher or lower sample rates. Behavioral cohorts are dynamically refreshed every few minutes. However, there is a small window where a brand new user or company has not been enrolled into a cohort yet and have their rates propagated to your server integration. This causes the user/company to fall back to global and regex-based sampling rules.

To set a sample rate for a cohort of users or companies:

1. Save a behavioral cohort

If you have not created a behavioral cohort (which drives sampling rules, governance rules, among other things), you will need to create one.

  • Click the orange new button at top left, and select User Cohort_ or Company Cohort.
  • Add the customer demographics or behavioral filters you want for the cohort. Once done, click the _Create Cohort_button at top left and give it a name.

In this example, we are selecting users who were created in the last 4 months and performed at least 1 API call to either /widgets/buying or /purchases in the last year.

Create User Cohort

2. Set the sample rate for the cohort

Once you have created a cohort, go back to Dynamic Sampling from the bottom left menu.

Click the orange + Rule button and select New User Rule or click New Company Rule based on what you created in step 1.

Add Sampling Rule for Dynamic Sampling

From here, you will see a list of saved behavioral cohorts. Select one and then set the sample rate.

Apply Sample Rate to User Cohort

In this case, 100% means log all API calls, whereas 0% means suppress all data collection.

Static rate for a single user or company

Finally, any user or company can have a defined sample rate from 0% to 100%. By default, all users sample 100% of API traffic (i.e. all traffic). Reducing any customer to 0% effectively suppresses all data collection for that user (A mechanism to enable GDPR/CCPA compliance). A use case of leveraging user level sampling is to set your 10 largest customers to a less than 100% sample rate, will collect 100% of API traffic for all others.

To set the sample rate for an individual user or company, click on Users or Companies in the top header, then look up the user or company of interest. Once you open up their profile view, click the orange button Edit Sample Rate.

Regex rules

With Regex rules, you can dynamically adjust sample rates based on specific fields within the API call. For example, you may want to log any response status code > 400 at 100%. You can also use regex rules to suppress data collection on uninteresting traffic such as GET /health/probe. API Calls do not need to be attributed to a user or company for regex rules to work which make them perfect for health probes and test traffic.

To get started, click your avatar at the bottom left, then select the Dynamic Sampling menu option.

Dynamic Sampling Menu

Click the orange + Rule button and select New Regex Rule. This will open a panel where you can select the fields and the regular expression to match.

Add Sampling Rule for Dynamic Sampling

Select the field to match on such as the Request Route or the Response Status. Then, add your regular expression. If you need help, view regex101 to build one.

Add Regex Rule for Dynamic Sampling

Then select the sample rate that should be applied for this regular expression. In the above example, we want to suppress data collection for non-interesting traffic by not logging health probes.

Global sampling

The global sample rate is used by any user (or company) that doesn’t have its own sample rate nor belongs to any behavioral cohort. To modify the global sample rate, click on “Global Sampling” button

Global Sample Rate

This also brings up a few other options to suppress traffic such as on IP address or known bots and crawlers.

Auditing effective sample rates

There are two ways to audit the effective sample rate for a customer. The effective rate include both any statically defined sample rate for a user/company such as for GDPR compliance and also any sample rates dynamically set by behavioral cohort rules. It does not include non-customer centric rates like regex rules which are applied on a per-request level. To view effective sample rate for a customer:

  1. Going to any user or company profile and view the Effective Sample Rate section.

  2. Select Dynamic Sampling from under the Alerts & Governance header bar. Then click the orange Settings button at the top right. This will open a pop up like below:

Effective Sample Rates

The time when your cohort rules were last updated is also displayed. This means the most recent time when either you manually saved the cohort criteria or because a user/company was added or removed from a cohort based on their behavior. If the cohort hasn’t been updated, this could imply no customers have been added or removed recently.

SDK/Plugin support for dynamic sampling

SDK User Rule Company Rule Regex Rule
AWS Lambda Node Y Y N
AWS Lambda Python N N N
AWS Lambda GO N N N
Azure APIM Y Y N
Cloudflare Y Y N
C# .NET Y Y N
Envoy Y Y N
Go Y Y N
Java Servlet Y Y N
Kong API Gateway Y Y Y
K8s NGINX Ingress Y Y N
NGINX Y Y Y
Node.js Y Y N
PHP Laravel N N N
PHP Slim Y Y N
PHP Symfony N N N
Python Django Y Y N
Python Tornado Y Y N
Python WSGI Y Y N
Python ASGI Y Y N
Python Requests Y Y N
Ruby Rack Y Y N
Tyk API Gateway Y Y N
3Scale Y Y N
Play Filter Y Y N

Updated: