Using AI-powered API Analytics with Kong API Gateway
Kong is a popular open-source API gateway to help manage your APIs. With Kong, you can handle authentication, rate limiting, data transformation, among other things from a centralized location even though you have multiple microservices. Kong is built on NGINX at it’s core, one of the most popular HTTP servers.
Being open-source, Kong is very easy to deploy on-premises without requiring the installation of many components other than a Postgres or Cassandra store. In fact, you can be running in just a few minutes.
Moesif has a plugin available in the Kong hub that enables you to understand API usage and monitor your API traffic. Like Google Analytics, Moesif is a hosted service so you don’t have to worry about maintaining large data pipelines and data stores while gaining the similar insights you would expect from a Mixpanel or Google Analytics. With Moesif, you’re able to understand how your API is used and by what customers, identify which customers are running into integration issues, and monitor for endpoints that need optimization.
Overview
Moesif Kong plugin is an agent that collects metrics and sends to the Moesif collection network. This enables you to get a complete picture of your API usage even across different Kong instances and data center regions. Moesif recommends using the same Moesif Application Id for all services and routes configured in Kong. However, it’s recommended to use separate Moesif Application Ids for each isolated environment such as a production and development environment.
Installation
To install the plugin, install the Lua Rock and then enabling the plugin:
luarocks install --server=http://luarocks.org/manifests/moesif kong-plugin-moesif
You may have to update your kong.conf by appending moesif to the plugins field (or custom_plugins if old version of Kong). Make sure the field is not commented out.
plugins = bundled,moesif # Comma-separated list of plugins this node
# should load. By default, only plugins
# bundled in official distributions are
# loaded via the `bundled` keyword.
Now that the plugin is installed, enable the plugin.
curl -i -X POST --url http://localhost:8001/plugins/ --data "name=moesif" --data "config.application_id=Your Moesif Application Id";
Once you enable the Moesif plugin, data should start showing up in your Moesif dashboard.
Below, I am pulling up API usage over time grouping by the request User-Agent.
We can see that the Node SDK is the most popular SDK for this API.
How the plugin works
The plugin captures metrics locally and queues them on your local Kong instance. This enables the plugin to send metrics data to the Moesif collection network out of band without impacting your app. For best practices in designing SDKS, take a look at this post.
The plugin is open source and available on GitHub.
Supported functionality
Majority of functionality that is available when using a Moesif SDK is also available when using the Kong plugin. Specifically:
Service/route detection
Moesif enriches the data so that it is better organized. For example, instead of filtering on
GET /service_a/items/1 OR GET /service_a/items/2 OR GET /service_a/items/3, Moesif will detect the REST
templates including any parameters and ids enabling a single filter on GET /service_a/items/:id.
Similarly, if you’re using Kong for a GraphQL API, that is also supported.
Geo segmentation
The Kong plugin will capture IP data from NGINX in addition to timing metrics to enable geo heatmaps and filtering such as the one below showing where users experience a latency greater than one second.
User sessions
If you configured Kong with authentication, the Moesif Kong plugin will automatically captures the authenticated credentials, no Configuration is needed for this. Capturing session information enables Moesif to generate API session traces and user profiles along with information on user first seen time, user last seen time, etc.
You may find it beneficial to add user demographic info such as customer email and first name. As long as you have the user’s API key/session token, you can use any of the Moesif API libs to add/update user profile metadata at any time.
This enables you to pull up the full customer journey for one of your API consumers. You’ll see demographic info for the customer along with any API activity they made.
Because of this, Moesif is able to perform more advanced user cohort analysis, which allows you to track the 30-day active retention over time. Retention measures the percent of users within a cohort that return and stay active with your product. What is considered active for your product depends on the type of product. For a streaming mobile app, being active on a day may mean playing a song. For a payments API, it could be processing a credit card payment on a day.
Incoming and outgoing APIs
The Moesif Kong plugin can be used for your own APIs along with outgoing APIs to your partners that you depend on. This can be very useful to understand what’s happening with your partner’s infrastructure and know of any issues. As long as traffic is flowing through your Kong instance, Moesif can analyze it.
Conclusion
In this way, this plugin will capture API requests and responses and log to Moesif for easy inspecting and real-time debugging of your API traffic via Kong and Kong handling all the other services around the application.
Related Articles
API Analytics and Monitoring
Monitoring Cost and Consumption of AI APIs and Apps
Discover how Moesif can help manage and optimize your AI-driven APIs with powerful observability tools that track costs and usage efficiently.
API Development
Unlock Deep API Insights with Moesif’s HTTP Body Analytics
Learn how to use Moesif to analyze HTTP bodies.
API Analytics and Monitoring
How Moesif API Observability Compares to Treblle
Learn how Moesif’s API observability platform compares to Treblle.
Podcasts
From Vision to Venture Ep. 03: Gregory Koberger, Founder ReadMe.
In today’s episode, Gregory Koberger, founder of ReadMe, an API documentation and developer tools platform. Greg shares the highs and lows of building ReadMe.