Using AI-powered API Analytics with Kong API Gateway

Kong is a popular open-source API gateway to help manage your APIs. With Kong, you can handle authentication, rate limiting, data transformation, among other things from a centralized location even though you have multiple microservices. Kong is built on NGINX at it’s core, one of the most popular HTTP servers.

Being open-source, Kong is very easy to deploy on-premises without requiring the installation of many components other than a Postgres or Cassandra store.

In fact, you can be running in just a few minutes. However, this also prevents Kong from creating deep insights from your API traffic.

Moesif has a plugin available in the Kong hub that enables certain API data to be sent over to Moesif for deeper analysis. With Moesif, you’re able to understand how your API is used and by what customers. You can use Moesif to perform A-B testing on your APIs, and identify which endpoints needs optimization.

Overview

Moesif unifies all your API data together automatically in a single holistic view even if individual Kong instances are running in different data centers behind dynamic DNS such as Amazon Route 53. No need to manually cluster the instances across data centers. Moesif recommends using the same Moesif Application Id for all services and routes configured in Kong so you have a unified view for analytics.

However, it’s recommended to use separate Moesif Application Ids for each isolated environment such as a production and development environment. More info on Moesif orgs and apps.

Because the data capture is done via the plugin, no extra SDKs or agents need to be installed in your app nor does your app need to be redeployed.

It’s as simple as installing the Lua Rock and then enabling the plugin:

curl -i -X POST --url http://localhost:8001/services/{service}/plugins/ \
     --data "name=moesif"   \
     --data "config.application_id=X-MOESIF-APPLICATION-ID";

Once you enable the Moesif plugin, data should start showing up in your Moesif dashboard. Below, I am segmenting API data by request User-Agent and filtering by traffic from the United Kingdom. We can see that the Node SDK is the most popular SDK for this API in the UK.

Enterprise reliability

Moesif has a reverse CDN that captures data from your Kong instance around the world. This ensures your data can always be captured even if a datacenter is offline. In addition, the Kong plugin does not reroute your API traffic through any additional proxies. The plugin is like an agent that captures API data out-of-band similar to how Google Analytics captures clickstream data from your websites via their Javascript SDK.

The plugin is entirely open source and available on GitHub.

Supported functionality

Majority of functionality that is available when using a Moesif SDK is also available when using the Kong plugin. Specifically:

Service/route detection

Moesif enriches the data so that it is better organized. For example, instead of filtering on GET /service_a/items/1 OR GET /service_a/items/2 OR GET /service_a/items/3, Moesif will detect the RESTful templates enabling a single filter on GET /service_a/items/:id. Similarly, if you’re using Kong for a GraphQL API, that is also supported.

IP/Geo data

The Kong plugin will capture IP data from NGINX in addition to timing metrics to enable geo heatmaps and filtering such as the one below showing where users experience a latency greater than one second.

User sessions

If you configured Kong with authentication, the Moesif Kong plugin will automatically captures the authenticated credentials, no Configuration is needed for this. Capturing session information enables Moesif to generate API session traces and user profiles along with information on user first seen time, user last seen time, etc.

You may find it beneficial to add custom metadata to user profiles such as customer email and first name. As long as you have the user’s API key/session token, you can use any of the Moesif API libs to add/update user profile metadata at any time.

As usual, any API calls captured in Moesif can be exported to Postman to be replayed.

Incoming and outgoing APIs

The Moesif Kong plugin can be used for your own APIs along with outgoing APIs to your partners that you depend on. This can be very useful to understand what’s happening with your partner’s infrastructure and know of any issues. As long as traffic is flowing through your Kong instance, Moesif can analyze it.

Conclusion

In this way, this plugin will capture API requests and responses and log to Moesif for easy inspecting and real-time debugging of your API traffic via Kong and Kong handling all the other services around the application.

Moesif is the most advanced API Analytics platform, supporting REST, GraphQL, Web3 Json-RPC and more. Thousands of API developers process billions of API calls through Moesif for debugging, monitoring and discovering insights.

Learn More