Usage-Based API Pricing with Moesif and Recurly

Back in the day, it was normal to buy or rent servers and pay for them in a lump sum, independent of utilization. With the rise of serverless computing, infrastructure to build APIs has become cheaper, and on-demand pricing is the norm. Now that you only pay for what you use, why not include usage-based pricing as a feature in your own SaaS offering.

With usage-based pricing your customers don’t get intimidated by high upfront costs, instead their costs grow as their business grows. Recurly is a subscription management service that allows you to charge your customers based on their API usage.

This guide assumes you’re already familiar with Moesif’s Metered API Billing and Recurly’s Subscription Management. We also recommend reading A Playbook to Properly Implement Pay As You Go Pricing to become familiar with different billing models.

This article explains how to set up usage-based pricing using Recurly in a Node.js Express install, with the API usage figures coming from Moesif.

How the solution works

In order to properly implement usage-based billing, you need a few components beyond standard subscription management:

  • Meter each customer’s platform usage and invoice based on that usage
  • Send automatic reminders to customers when they approach or exceed a quota or threshold
  • Provide a dashboard for a customer to inspect their subscription usage

Prerequisites

You’ll need a Moesif account and a Recurly account. You also need to have Node.js installed.

Setting up Moesif

The first step is linking API requests with your customer id. This is a basic step in Moesif’s setup and is needed later to tell Recurly what a customer has to pay for.

// 1. Import Modules
const express = require('express');
const app = express();
const moesif = require('moesif-nodejs');

// 2. Set the options, the only required field is applicationId
const moesifMiddleware = moesif({
  applicationId: 'Your Moesif Application Id',

  // Optional hook to link API calls to users
  identifyUser: function (req, res) {
    return req.user ? req.user.id : undefined;
  },
});

// 3. Enable the Moesif middleware to start logging incoming API Calls
app.use(moesifMiddleware);

You also have to replace identifyUser with a function that extracts the user from the request context. In this example, the id is set by Node.js authentication middleware. If you authenticated at the company level, you could choose to implement the identifyCompany function instead. With this setup, all your API requests will be logged to Moesif API Analytics. The next step is to set up Recurly.

Setting up Recurly

Recurly has plans and customer accounts, where customers can subscribe to more than one plan. In this example, we’ll create a simple plan for our usage-based pricing and the customer can only be subscribed to the single plan.

To create a usage-based plan in Recurly, create the plan and then choose a usage-based add-on. You can set a base price and then increase it, on-demand, when customer use dictates. We will create a free plan and only charge per use in this example, to keep things simple.

Figure 1 shows where to find the configuration of the Recurly plans.

Figure 1: Recurly plans


Click on “New Plan” in the top right and then create a new plan called “Usage-Based” with a plan code of “001”. The price per billing period will stay zero.

Under Plan Add-Ons, create a new add-on. This add-on needs a name and a code. It should be billed at the end of a billing cycle, because we don’t know what a customer will use in advance. Next, create a new “measure unit” called “requests” and set it to a fixed price of $0.01. That way, every request costs a cent. The add-on should not be optional.

Your plan creation form should look like the following:

Figure 2: Recurly plan creation form

Linking Moesif and Recurly

Now that we have a usage-based plan in Recurly, customers can subscribe to it. We need to link the subscription ID in Recurly to the customer id in Moesif.

Recurly explicitly states in their docs that you shouldn’t use webhooks for account creation, but use instead the return values from their API client. This means Recurly subscription creation and Moesif company and user creation have to be done in one go.

Let’s start with the Recurly part. Inside an Express API endpoint for a signup, you have to use the following two Recurly API calls.

  let account;
  try {
    account = await recurlyClient.createAccount({
      code: getNewCustomerId(),
      firstName: request.body.firstName,
      lastName: request.body.lastName,
      address: request.body.address,
    });
  } catch (error) {
    console.log(error);
    return response.end();
  }

  let subscription;
  try {
    subscription = await recurlyClient.createSubscription({
      account,
      addOns: [{ code: "001" }],
      currency: "USD",
      planCode: "001",
    });
  } catch (error) {
    console.log(error);
    return response.end();
  }

The first one will create a Recruly account for your user, and the second one will start a subscription. You need to create a unique account code on your own. The easiest way to do this is to sign up a customer to your own authentication system first and use the ID you generated.

The second Recurly API call will subscribe the Recurly account to the plan. The plan’s code and its add-on code are the ones we used when we created the plan before.

If everything went well, you should end up with two objects, account and subscription. We will use these in the next step to link them with a company and user inside Moesif.

const moesifCompany = {
  company_id: subscription.uuid,
  metadata: subscription,
};

const moesifUser = {
  user_id: account.account_code,
  company_id: moesifCompany.company_id,
  email: account.email,
  first_name: account.first_name,
  last_name: account.last_name,
  metadata: { ...account },
};

moesifapi.ApiController.updateCompany(
  moesifCompany, (error) => {
  if (error) console.log(error);
};

moesifapi.ApiController.updateUser(moesifUser, (error) => {
  if (error) console.log(error);
};

To create a Moesif company from a Recurly subscription, we have to create a new object with a company_id; we will use the uuid of our freshly created subscription for this and put the rest of the subscription data into the metadata of our moesifCompany object.

Next, we use the moesifapi library, which will be installed alongside moesif-nodejs. The updateCompany function will create a new company record in the Moesif service. Since we used the subscription’s uuid, these two are now linked.

The final stage is to create a Moesif user. While it isn’t required for billing, which works on a company level, it will help later on within Moesif if we want to find out who issued which request for the company.

Handle API Billing with Moesif

Learn More

Sending API Usage to Recurly

Pushing metrics vs prerenewal wehook

There are two different ways to sync usage from Moesif to Recurly

  1. Sending Recurly the billing period’s usage when the Subscription Prerenewal webhook is triggered, which is sent 1 day before a subscription renewal. Less work, but less accurate.
  2. Continuously send Recurly usage data such as in hourly or daily jobs. More work but also more accurate

Company usage query

In order to determine a customer’s usage, we first need to decide what API calls are “billable” and how much each is worth. For example, a communications API may charge based on SMS sent whereas an e-commerce platform may charge based on purchases made. We can leverage Moesif’s advanced filtering mechanism and Management API to build this query in a few clicks. In this example, we are including all API calls where the route matches /purchases or /withdraws.

  • To generate the JSON in the getCompanyUsageQuery() function, log into Moesif and go to Events -> Segmentation.
  • From here, add any filters that should be included such as specific endpoints or fields.

Create Management API query

  • Be sure to add a filter on a company id. The actual value and the chart’s date range don’t matter as they will be overridden in the code.
  • Click the orange “Embed” button at the top right and then “Get Via API”. This will provide the full JSON query to use for that report.
  • The query params from and to along with the payload value company_id.raw will be will be overridden in the code below.

Get Via API

To sync the Recurly subscriptions to Moesif, we’ll fetch all the recently created or updated subscriptions from the last 24 hours using a recurring cron job. The Recurly Subscription Id will map to the Moesif Company id where as the Account Id will map to a Moesif User Id.

The code to handle usage-based invoicing

const recurly = require('recurly-nodejs');
const cron = require('cron');
const basicAuth = require('express-basic-auth');
const bodyParser = require('body-parser');
const express = require('express');
const safeGet = require('lodash/get');
const superagent = require('superagent');
const moesifapi = require('moesifapi');
const moment = require('moment');

require('body-parser-xml')(bodyParser);

const app = express();

const UNIT_COST_IN_CENTS = 1; // How much each transaction is worth in cents

// Simple sample query which can be found going to "Events" -> "Segmentation" in Moesif.
// Then select the orange Get Via API button under "Embed".
function getCompanyUsageQuery(companyId) {
    return {
        aggs: {
            seg: {
                filter: {
                    bool: {
                        must: [
                            {
                                terms: {
                                    'request.route.raw': [
                                        '/purchases',
                                        '/withdraws',
                                        '/'
                                    ]
                                }
                            },
                            {
                                term: {
                                    'company_id.raw': companyId
                                }
                            }
                        ]
                    }
                },
                aggs: {
                    weight: {
                        sum: {
                            field: 'weight',
                            missing: 1
                        }
                    }
                }
            }
        },
        size: 0
    }
}

app.use(bodyParser.xml());
app.post('/recurly/webhooks', (req, res) => {
    const event = req.body;
  
    if (event && event.prerenewal_notification) {

          // We should query metric the previous billing period.
          const params = {
              from: moment(prerenewal_notification.subscription.current_period_started_at).toISOString(),
              to: moment(prerenewal_notification.subscription.current_period_ends_at).toISOString()
          };

          console.log(JSON.stringify(params));

          // Get usage from Moesif
          return superagent
              .post(`https://api.moesif.com/v1/search/~/search/events`)
              .set('Authorization', `Bearer ${process.env.MOESIF_MANAGEMENT_API_KEY}`)
              .set('Content-Type', 'application/json')
              .set('accept', 'json')
              .query(params)
              .send(getCompanyUsageQuery(subscription.id))
              .then((countResult) => {

                  const count = safeGet(countResult, 'body.aggregations.seg.weight.value');

                  console.log(`Received count of ${count}`)
                  const amount = count * (UNIT_COST_IN_CENTS); // How much each transaction is worth in cents

                  console.log(`Adding cost for subscription=${subscription.id} of ${amount}`);

                  // Save the API usage amount to Recurly
                  recurly.createUsage(
                    prerenewal_notification.subscription.uuid,
                    "001",
                    { amount: amount }
                  );
              }).catch((error) => {
                  console.log(error.text);
                  res.status(500).json({ status: 'internal server error' });
              });
    } 
    console.log(JSON.stringify(event));
    res.status(500).json({ status: 'internal server error' });
});

app.listen(process.env.PORT || 5000, function() {
    console.log('moesif-recurly-example is listening on port ' + (process.env.PORT || 5000));
});

Embed Usage Reports

Now that your usage-based invoicing is fully set up with Moesif and Recurly, you’ll also want to look into ways to keep your customers informed of their usage. The best way we recommend you to do this is to use embedded dashboards, which elegantly shows usage metrics.

Embedded usage reports enable you to provide self-serve metrics to your customers, so that they can understand what they’re paying for. These reports create additional transparency and trust with your customers, since they can easily track the value they get relative to cost. To get started, view embedded template docs and the example GitHub project.

Set up Quota Alerts and Emails

You can also leverage Moesif Behavioral Emails to automatically inform customers when they are close to, or exceeding, their quota for the billing period. This can be a static threshold, specific to a plan, or even specific to groups of customers.

To get started, follow our guide to set up automatic notifications of quota and billing issues.

Summary

Moesif and Recurly are an excellent combination for usage-based billing. The telemetry is stored in Moesif, so you can leverage its additional widgets like behavioral emails and embedded dashboards to keep your customers informed of their subscription.

You can also set up alerts with Moesif, so your own customer success team can proactively reach out if usage/billing wildly fluctuates during a billing cycle.

Make Your API Platform Successful With Moesif

Learn More
Usage-Based Billing with Moesif and Recurly

Usage-Based Billing with Moesif and Recurly

Learn More