Creating Alert Rules

This guide assumes you are familiar with Alerts already. If not, please review API Monitors & Alerts

Example creating an alert rule

Alert rules can be created on any time series chart in Moesif. The below example walks through creating a static (threshold-based) alert rule for 90th percentile latency is over 1 second. Since it’s important to track each customer’s experience independently, the example also adds a Group By on company.Company Domain.

Creating Alert Rules

  1. Go to Events -> Time Series from the top menu to start creating a new report.

  2. Configure your chart’s filters, metric, and time period for what you would like to monitor.

  3. Click the orange Alert button at the top right to open the alert panel.

  4. In the panel that opens on left side, configure your static threshold or dynamic sensitivity (for anomaly detection). Here, you can give your alert a name and also pick which channels to send notifications to.

  5. Click create, a confirmation dialog is displayed detailing how the rule is triggered.

Alert options

Chart Filters

Any filters you added to your time series chart will also be applied to the alert rule. For example, to only alert when the response.status is 500 Internal Server Error, add a filter like so:

Select only 500 errors

Chart Group By

By adding a group by to your chart, Moesif will create a multi-dimensional alert which means Moesif will monitor each of the group by values separately. This makes it easy to create a single alert rule that monitors hundreds of different metrics without manually creating hundreds of different alert rules. As an example, you may want to track a metric separately for each customer or each URI Route.

Let’s say we want to get alerted when a specific customer has a spike in latency, our alert rule would look like so:

Creating Alert with Group By

The alert criteria will be presented in the blue box shown above.

Chart metric

The metric plotted in the chart is what Moesif will monitor. You can select one of the predefined metrics or build your own. For example, we can create an alert when a customer sends an unusually large amount of data to our API which could imply abuse or a hacker like so:

Creating Alert with Custom Metric

In this case we’ll keep our group by user.email so we track each user’s upload amount separately.

Chart interval

Alerts require an interval for the metric aggregation. For example, if you select event count as your chart metric and the chart is currently set to 15 minutes, then your alert will be tracking the event count over rolling 15 minute windows.

Hourly or smaller intervals are evaluated continuously every minute in overlapping rolling windows. For example if you select 15 minutes, Moesif will aggregate the metric from the last 15 minutes with a series like so: 12:00 to 12:15, 12:01 to 12:16, 12:02 to 12:17, and so on. On the other hand, daily or larger intervals are evaluated once per calendar interval. This means if you select daily, Moesif will aggregate your metric from 12:00 AM to 11:59pm and send a daily report. Your calendar and timezone settings can be changed under Apps and Team settings.

Alert Rule Settings

In addition to the chart conditions rules, there will be additional settings to configure once you click the Create Alert button.

Alert Name: Select a friendly name for your alert, which you can use to identify in emails and notifications.

Channels: Select the channels you would like to send the alert notifications to such as an email channel or Slack channel. If you already have channels created, you may select one here. If not you will need to set up new channels via the New Channel button.

Alert Type

Moesif supports two types of alerts. Static alerts enable you to define a specific threshold for when to trigger an alert. Whereas dynamic alerts leverages Moesif’s anomaly detection and triggers alerts when the metric deviates from the historical trendline.

Static Alert

Creating Static Alerts

Static alerts are simple alerts that trigger when your metric reaches your defined threshold for your metric. The alert will trigger once your metric (such as Event Count) reaches your threshold.

Static alerts supports >, <, >=, <=, and any as operators. If you want a recurring alert that triggers for every time interval regardless of metric value, select any under threshold settings.

Dynamic Alert

Creating Dynamic Alert

Dynamic alerts leverages Moesif’s anomaly detection which learns the trend line from historical data and alerts when the metric looks abnormal. This is very helpful when the metric varies drastically which would normally make static alerting impossible.

You can set the Direction field to monitor the changes that are important for you.

  • Increase will alert you when the monitored metric has an abnormal increase.
  • Decrease will alert you when the monitored metric has an abnormal drop off.
  • Both will alert you when the monitored metric has an abnormal change in either direction.

When used with group by, Moesif will monitor the trend line for each value separately. As example, let’s say you want to monitor when a customer has a drop of in API traffic so customer success can preempt any customer churn issue. In this case, group by user.email and set the dynamic alert to notify on decrease only. Moesif will alert you when a specific user’s API usage has a large decrease relative to the user’s historical trend line.

Changing dynamic alert sensitives

Moesif recommends the default sensitivities to start. If you feel that you’re receiving too few or too many false alerts, you can adjust the sensitivity.

Moesif has slow, medium, and fast detectors. If you’re receiving too many false alerts, drag slider right. If you want alerts for small deviations, drag the slider right.

Dynamic Alerts

Detector Alert Name When to decrease sensitivity
Abrupt spike detector SPIKE Your API receives a low volume of traffic so small fluctuations in error rates throws off the spike detector
Unusual change detector RISE/FALL  
Slowly trending higher detector POSITIVE TREND Your API has seasonal patterns (such as an API only used during working hours which naturally has an increase in error rates at 9am)

Advanced Settings

Advanced settings enables you to tweak when the notification are sent or silenced. For example, once an alert rule is triggered, you may want to silence it for a short period of time to avoid too many notifications.

Advanced Alert Settings

  • Silence notifications will silence any future notifications for the same alert rule for the period of time specified. This can help reduce the number of notifications for noisy alert rules.

  • Downtime window is a quiet period where alerts will not be sent. For example, if you have a cron job that performs some maintenance that brings down a service from 5AM to 6AM UTC everyday, you can set this schedule to avoid receiving false alerts.

Field Description Example
Start Time The start time for maintenance window in 24 hour clock time 05:00
End Time The end time for maintenance window in 24 hour clock time 06:00
Timezone The timezone to use for the start and end time PDT

Adding a Channel

A channel is a communication method to receive notifications and make it easier to ensure the right people receive the right alerts. Moesif supports the following channel types: email, sms, slack, pagerduty, and webhook.

Click the “New Channel” link to create one. More info on Channels.

Updated: