Kong is a popular open-source API gateway to help manage your APIs. With Kong, you can handle authentication, rate limiting, data transformation, among other things from a centralized location even though you have multiple microservices. Kong is built on NGINX at it’s core, one of the most popular HTTP servers. Being open-source, Kong is very easy to deploy on-premises usually in just a few minutes without requiring the installation of many components other than a Postgres or Cassandra store.
Kong Ingress Controller implements authentication, transformations, and other functionalities across Kubernetes clusters with zero downtime. Kong Gateway connects Kubernetes clusters with services running across any environment or platform. By integrating with the Kubernetes Ingress Controller, Kong ties directly to the Kubernetes lifecycle. As applications are deployed and new services are created, Kong will automatically live configure itself to serve traffic to these services. Kong and the Kong Ingress Controller has a full management layer and API, live configuration of targets and upstreams, and a durable, scalable state storage using either Postgres or Cassandra that ensures every Kong instance is synced without delay or downtime.
Moesif provides a plugin for Kong that makes getting started with API observability in just a few minutes so you can stay focused on shipping features customers love rather than deal with the maintenance cost of building your own data infrastructure. With Moesif, you’re able to understand how your API is used and by what customers, identify which customers are running into integration issues, and monitor for endpoints that need optimization. In this article, we’ll talk about the insights Moesif gives you, and how to integrate it with Kong Ingress Controller. Then, we will talk about how to best leverage API observability using Moesif and Kong.
Moesif Kong plugin is an agent that collects metrics and sends to the Moesif collection network. This enables you to get a complete picture of your API usage even across different Kong instances and data center regions. Moesif provides deep insights for engineering teams to understand how their APIs are used and quickly troubleshoot complex issues. Because Moesif also tracks who is calling your API and how they are accessed, product-driven teams can understand the entire customer journey and where to invest more resources. With API observability, forward-thinking engineering leaders can empowers customer-facing teams with self-service analytics on activation funnels, retention reports, and more. Moesif also analyzes your API payloads for troubleshooting and business insights so you’re able to understand utilization of specific payload keys, etc.
Setting up the Kong Ingress Controller
Setup Kong for Kubernetes
Setting up Kong for Kubernetes using Minikube is as simple as:
kubectl apply -f https://bit.ly/k4k8s
Setup environment variables
You’ll setup an environment variable with the IP address at which Kong is accessible. This will be used to actually send requests into the Kubernetes cluster. You will create a tunnel for kong-proxy and get the url.
minikube service --url kong-proxy -n kong export PROXY_IP=<URL from the above step>
You could test connectivity to Kong using -
curl -i $PROXY_IP
Add the Moesif plugin
You’ll need to download the kong-moesif-plugin and navigate to kong/plugin directory to create a configMap using
kubectl create configmap kong-plugin-moesif --from-file=moesif -n kong
Please ensure that this is created in the same namespace as the one in which Kong is going to be installed.
Next, you’ll need to edit Kong’s Deployment to load Moesif plugin using
kubectl edit deployment.v1.apps/ingress-kong -n kong
Patch which needs to be applied to the existing deployment (truncated)
apiVersion: apps/v1 kind: Deployment metadata: name: ingress-kong namespace: kong spec: template: spec: containers: - name: proxy env: - name: KONG_PLUGINS value: moesif - name: KONG_LUA_PACKAGE_PATH value: "/opt/?.lua;;" volumeMounts: - name: kong-plugin-moesif mountPath: /opt/kong/plugins/moesif volumes: - name: kong-plugin-moesif configMap: name: kong-plugin-moesif
Once you patch the deployment, you’ll need to rollout the Kong Ingress Controller using -
kubectl rollout status deployment.v1.apps/ingress-kong -n kong
Setup KongPlugin resource
To setup KongPlugin resource, you will need your Moesif Application Id. You can get one by signing up for a free Moesif account, then select Kong during the onboarding. Moesif recommends using of a single application Id for all Kong instances and data-center regions. This ensures you have a unified view of your API usage data regardless of physical topology. You can still break down by any number of attributes using Moesif’s high-cardinality, high-dimension analytics engine.
Moesif still recommends creating separate Application Ids for each environment such as Production, Staging, and Development to keep data isolated.
echo " apiVersion: configuration.konghq.com/v1 kind: KongPlugin metadata: name: moesif disabled: false config: application_id: Your Moesif Application Id debug: false plugin: moesif " | kubectl apply -f -
Create a new Ingress resource which uses Moesif plugin
You can annotate an Ingress or Service resource to instruct Kong on when to execute the plugin. You could configure plugins globally, on Service resource, or for a specific consumer.
echo " apiVersion: extensions/v1beta1 kind: Ingress metadata: name: demo-example-com annotations: konghq.com/plugins: moesif kubernetes.io/ingress.class: kong spec: rules: - host: example.com http: paths: - path: /bar backend: serviceName: echo servicePort: 80 " | kubectl apply -f -
Send a Request
Once you have the Moesif plugin installed, your API traffic should start showing up in the live event stream within Moesif. You could send traffic to the resource configured using
curl -i -H "Host: example.com" $PROXY_IP/bar/sample
How to use API observability
The first thing you’ll probably interested in is engineering metrics like how is my API performing. This type of metric can be pulled up by going to Events -> Time Series view within Moesif. Then you can select 90th percentile latency as the metric to plot. You can then group by the URI route to understand which endpoints have the worst performance. Here, you can filter your traffic by API attributes like route, verb, along with HTTP headers and body fields.
Handling sensitive data
If your application consists of sensitive data such as healthcare or financial data, you can become data compliant with one of two options:
1. Zero-knowledge security
Client-side encryption has the benefits of low-maintenance SaaS while still putting you in control of your data. Because you control and encrypt the data on-premises before being sent to Moesif, Moesif physically cannot access your data. This requires only running a small appliance within your infrastructure called secure proxy to handle encryption/decryption of your data on the fly.
2. Data masking
If you don’t want client-side encryption, you can also mask data directly using the plugin. This is handled via the
config.response_body_masks configurations option. For example, you could mask a field called password that might be present in the payload. Additionally if you want to remove logging request and response body all together, you could set the
config.disable_capture_response_body configuration option to
In this way, Moesif plugin will capture API requests and responses and log to Moesif for deep API observability and real-time monitoring of your API traffic via Kong and Kong handling all the other services around the application.