,

How to Monitor NGINX API Logs with Moesif

With increasing popularity among API Gateways, which provides authentication, rate-limiting and access control between applications that expose their APIs and the consumer of the APIs are the bedrock of API infrastructure. Simple architecture which bundles API related functionality inside a single component provides load balancing, caching and ability to scale to ensure high availability.

Nginx is a high‑performance, highly scalable, highly available web server, reverse proxy server, and web accelerator (combining the features of an HTTP load balancer, content cache, and more). NGINX has a modular, event‑driven, asynchronous, single-threaded architecture that scales extremely well on generic server hardware and across multi-processor systems.

OpenResty is a full-fledged web application server by bundling the standard NGINX core, lots of 3rd-party NGINX modules, as well as most of their external dependencies. It comes with a range of features, such as load balancing, content cache, and many more.

Which RESTful endpoints have high 90th percentile latency?

Overview

Moesif Nginx OpenResty plugin is an agent that capture metrics and sends to the Moesif collection network. This enables you to get a complete picture of your API usage even across different instances and data center regions. The plugin captures metrics locally and queues them which enables the plugin to send metrics data to the Moesif collection network out of band without impacting your app. Moesif recommends using the same Moesif Application Id for all services and routes configured in Nginx. However, it’s recommended to use separate Moesif Application Ids for each isolated environment such as a production and development environment.

In this article, we’ll talk about the insights Moesif gives you, and how to integrate it into your Nginx OpenResty setup.

Setting Up Moesif API Analytics with Nginx OpenResty

Moesif has a plugin available in the Luarocks that enables you to understand API usage and monitor your API traffic. With Moesif, you’re able to understand how your API is used and by what customers, identify which customers are running into integration issues, and monitor for endpoints that need optimization.

The recommended way to install Moesif is via Luarocks:

luarocks install --server=http://luarocks.org/manifests/moesif lua-resty-moesif

How to use

Edit your nginx.conf file to configure Moesif OpenResty plugin: Replace /usr/local/openresty/site/lualib with the correct plugin installation path, if needed.

lua_shared_dict moesif_conf 2m;

init_by_lua_block {
   local config = ngx.shared.moesif_conf;
   config:set("application_id", "Your Moesif Application Id")
}

lua_package_path "/usr/local/openresty/luajit/share/lua/5.1/lua/resty/moesif/?.lua;;";

server {
  listen 80;
  resolver 8.8.8.8;

  # Default values for Moesif variables
  set $user_id nil;
  set $company_id nil;

  # Optionally, identify the user and the company (account)
  # from a request or response header, query param, NGINX var, etc
  header_filter_by_lua_block  { 
    ngx.var.user_id = ngx.req.get_headers()["User-Id"]
    ngx.var.company_id = ngx.req.get_headers()["Company-Id"]
  }

  access_by_lua_file /usr/local/openresty/luajit/share/lua/5.1/resty/moesif/read_req_body.lua;
  body_filter_by_lua_file /usr/local/openresty/luajit/share/lua/5.1/resty/moesif/read_res_body.lua;
  log_by_lua_file /usr/local/openresty/luajit/share/lua/5.1/lua/resty/moesif/send_event.lua;

  # Sample Hello World API
  location /api {
    add_header Content-Type "application/json";
    return 200 '{\r\n  \"message\": \"Hello World\",\r\n  \"completed\": true\r\n}';
  }
}

Supported Functionality

Moesif enriches the data so that it is better organized. For example, instead of filtering on GET /service_a/items/1 OR GET /service_a/items/2 OR GET /service_a/items/3, Moesif will detect the REST templates including any parameters and ids enabling a single filter on GET /service_a/items/:id.

Mask Sensitive Data

Moesif provides the ability to remove any sensitive data in the HTTP headers or body or skip logging body payloads before sending to Moesif if you’ve privacy requirements or in a highly regulated industry.

To see the Nginx OpenResty integration with Moesif in action, you can git clone and run this example app from GitHub

Conclusion

In this way, this plugin will capture API requests and responses and log to Moesif for easy inspecting and real-time debugging of your API traffic via Nginx and Nginx handling all the other services around the application.

Get User-Centric API Monitoring for NGINX

Learn More
Get User-Centric API Monitoring for NGINX

Related Articles