Scripted Fields for Billing Meters

What Are Scripted Fields?

If a field does not exist in your request or response that you’d like to bill upon, you can create the field as a Scripted Field. A Scripted Field has the follow capabilities and limitations:

  • Write a script that retroactively creates a custom field using other fields, formulas, arithmetic, and more.
  • Only numbers, booleans, strings, and dates (treated as a number, milliseconds since epoch) are supported.
  • [field.path.field_name] is a reference to a field
  • If field value does not exists, for numbers it defaults to 0, and for dates it will default to epoch. You can use [field.path.field_name|50] to set default value.
  • Most mathematical functions are supported and if, else, then, end are supported. (detailed docs coming soon).
  • Only expressions are supported, no other variables or statements are supported.
  • Scripted fields with body fields may take several minutes to query.

Creating a Scripted Field

To add a Scripted Field, under the Metrics section on the Add/Update Billing Meter screen, click the Metrics dropdown and select Custom Metric > Select Field….

selecting a custom metric to bill upon in Moesif

Next, in the Field selector, select the Scripted Field option at the top.

select a scripted field in Moesif

In the modal that appears, you can input your Scripted Field expression. To add a field from Moesif into the expression, you can select it from the Add a reference to or freely type into the expression field. For more info on how to write an expression, check out the Using the Scripting Language section below.

scripted field modal in Moesif

Once your expression is input, click the Set button to add the field as a metric.

Using the Scripting Language

When using Scripted Fields, you create the new field by using MoesifFieldScript. MoesifFieldScript, also referred to as MSF, is a domain-specific language designed to perform custom computations on field values from events within the Moesif platform. This language provides the ability to evaluate logical, comparison, arithmetic, and functional expressions on event fields. Additionally, it supports accessing specific keys from the event data and provides default values if the specified key is not found.

Supported Constants, Operators, and Functions

MoesifFieldScript supports a variety of different mathematical constants, operators, and functions that can be used to create a value for a custom field. Below is list of currently supported entities.

Constants

type values
Numeric Constants Any whole number or decimal like 123, 4.56
Boolean Constants true or false

Operators

Arithmetic Operators:
symbol operator name/function
+ Addition
- Subtraction
* Multiplication
/ Division
Comparison Operators:
symbol operator name/function
== Equals
!= Not Equals
< Less Than
<= Less Than or Equal
> Greater Than
>= Greater Than or Equal
Logical Operators:
symbol operator name/function
and Logical AND
or Logical OR

Functions

Numeric Functions

symbol function name/functionality
round Rounds a number
ceil Rounds a number up
floor Rounds a number down
abs Absolute value of a number
min Minimum of numbers
max Maximum of numbers
avg Average of numbers
exp Exponential function
log Natural logarithm
pow Power of a number
sqrt Square root

String Functions

symbol function name/functionality
countSubstrings Count of substrings in a string

Examples

Below are some examples of how these operators and functions can be used within a MSF expression.

  1. To check if the value in a field, such as age, is less than 18:

     [age] < 18

  2. To sum the value of two fields, such as price and tax:

     [price] + [tax]

  3. To get the maximum of two fields, such as score1 and score2:

     max([score1], [score2])

  4. As a conditional expression, such as to check if status is active and return 1, otherwise 0:

     if [status] == "active" then 1 else 0 end

Referencing One Or More Fields in The Event

In Moesif, every API event is structured as an event document with various fields, including details about the request, response, and other associated metadata. When writing expressions in MoesifFieldScript, you can access these fields using a key path. This key path is a flattened representation of the JSON structure that omits array indices and focuses only on the names of JSON keys.

The schema of the event document is the same as what is provided in the event collector API. For more information on the schema, check out the API Call docs

To access specific values from the event data in MoesifFieldScript, you can use the following ways to access the value within a specific field using the fields key:

  • [keyName]: Accesses the value of the key keyName.
  • [key1.key2]: Accesses nested keys.
  • [keyName|defaultValue]: Accesses the key keyName and if not found, uses defaultValue.

Note: Special characters in key names should be escaped with a backslash (e.g. \\[ or \\\\).

Accessing Top-Level Fields

In Moesif, certain fields at the top level (those not contained in the request or response themselves), can also be accessed. For top-level fields in the event schema, such as time, user_id, and direction, MoesifFieldScript can access them directly by using their respective names. Below are two examples of such fields:

  • [time]: Accesses the timestamp of the request.
  • [user_id]: Accesses the associated user of the API call.

Accessing Request and Response Body Fields

Depending on how Moesif is configured, you may be able to use request and response body fields within your Scripted Field. For fields inside the body of the request and response objects, you can use the flattened JSON key path to access them. For example, consider an API event with the following request body:

{
  "request": {
    "body": {
      "fields": {
        "name": "John",
        "address": {
          "city": "San Francisco",
          "state": "CA"
        }
      }
    }
  }
}

To access the name field in MoesifFieldScript, you could use the following:

[request.body.fields.name]

To access the city inside the address object, you could use the following:

[request.body.fields.address.city]

Accessing Fields Within an Array

If there are arrays within the JSON structure, the flattened key path omits array indices. Instead, it focuses on accessing the nested keys directly. This simplifies access to fields and ensures consistency in expression writing. For example, Consider an API event with this request body:

{
  "request": {
    "body": {
      "fields": {
        "users": [
          {"id": 1, "name": "John"},
          {"id": 2, "name": "Jane"}
        ]
      }
    }
  }
}

To access the name of the second user, you would not use an array index. Instead, you’d access it as:

[request.body.fields.users.name]

It’s important to note that without array indices, this method can only access fields when the structure is known and consistent. If you need to access specific array elements, additional methods or processing might be required outside of MoesifFieldScript

Working With Dates

In MoesifFieldScript, date values are represented as the number of milliseconds that have elapsed since the Unix epoch (January 1, 1970, 00:00:00 GMT). This numeric representation allows for easy date comparison and arithmetic operations. Below are some examples of how dates can be used within a Scripted Field.

Checking If a Date is Before a Specific Time

To check if the date in the field fields.example.date is before January 2, 1970, 00:00:01 GMT (the Unix Epoch) or the value 1000, the timestamp for this specific time in milliseconds since epoch is 1000, the expression would be:

[fields.example.date] < 1000

Calculating The Difference Between Two Date Fields

If you have two date fields you’d like to find the difference between, you can simply use an equation in the expression to do so. For example, to find the difference in milliseconds between startDate and endDate fields, the expression would be:

[endDate] - [startDate]

Checking If an Event Occurred Within a Given Time Period

To return a boolean value of whether an event occurred within a given time period, you can calculate the difference between the current time and the event date and see if it is less than the time period you are concerned with. For example, given that there are 86,400,000 milliseconds in a day, to check if an event with the field eventDate occurred in the last 24 hours you could use the following expression:

[currentTime] - [eventDate] <= 86400000

Where currentTime is the current timestamp.

Adding Days/Time to a Date:

It is also possible to add days and time to a given date. For example, to add 7 days to a date value in the eventDate field, you can simply add 604800000 (the amount of milliseconds in one week) to the current value. That expression would lok like so:

[eventDate] + 604800000

Rounding Down to The Nearest Day:

If you want to ignore the time component and just consider the date, you can use the following expression to divide the timestamp by the number of milliseconds in a day, round down to get the number of days since the epoch, and then multiply the result by the number of milliseconds in a day to get the timestamp for the start of the day.

floor([eventDate] / 86400000) * 86400000

Working With Strings

Counting Substrings in a String:

If you want to count how many times a substring occurs in a string field, you can use countSubstrings().

countSubstrings([response.body.name|""], "AVG")

Support for Conditionals

In the scripted field, you will be able to add in conditionals, such as an if-else statement. Below are a few examples of how conditionals can be used

As a traditional, multi-line if-else statement:

if [request.query.billable] != true
then
1
else
2 * [response.body.usage]
end

or in a single-line statement:

if [field.value] == 42 then 2.2 * 31 else if [field.value] > 42 then  1.7 * 31 else 1.2 * 31 end end

Boolean operator’s can also be used within the conditional statement:

if [field.value] < 42 and [field.value] != 0 then 1 + 2 else 2 * 3 end

Another example is as part of a mathematical function, such as log:

log(if [request.body.pi] > 3 then
    [field.value]
else
    1.23
end)

Default Values

If a key being queried in an expression is not found, a default value can be provided. To specify a default value you can use the following syntax:

[user.age|25]

In this example, if user.age is not defined, the value used will be 25.

Updated: