Integration Guide - AWS API Gateway

Gain deep analytics from your AWS API Gateway access logs without any code changes or redeploy. This integration installs automatically via a CloudFormation template.

How it works

The integration works by Logging API calls to a Kinesis Data Firehose which sends access logs in batches to Moesif. Besides tracking typical API parameters like URI route and status code, the integration also tracks the user identity which makes them perfect for user behavior analytics tools like Moesif.

Diagram of Moesif AWS API Gateway logging architecture

Because your monitoring at the API gateway itself, you also gain visibility into rejected requests that never reach your underlying service such as AWS Lambda or EC2.

How to Install

1. Create a Kinesis Data Firehose

Click the below Launch Stack button to open CloudFormation which will automatically create a new Data Firehose configured with Moesif collection API as the destination.

Launch CloudFormation Stack

Ensure you checkbox I acknowledge that AWS CloudFormation might create IAM resources as shown below. Then click Create Stack.

AWS API Gateway Cloudformation create stack

If you experience errors or want to create the Firehose manually, follow these instructions

2. Enable API Gateway access logging

With the Firehose created, you need to configure API Gateway to send access logs to it.

  • Go to your AWS API Gateway instance within the AWS Console.
  • Select Stages on the left menu and then select the Logs/Tracing tab
  • Toggle on Enable Access Logging.
  • Add your Kinesis Firehose ARN created from Step 1 under Access Log Destination ARN.

3. Add the JSON log format

Under Log Format, add the below JSON snippet and ensure JSON format is selected. Moesif will safely ignore any extra keys.

It’s not possible to log the request and response body with this integration. If you require body analytics, choose a different integration such as a language-specific SDK

{   "apiId": "$context.apiId",
    "requestId": "$context.requestId", 
    "requestTime": "$context.requestTime", 
    "protocol": "$context.protocol", 
    "httpMethod": "$context.httpMethod",
    "resourcePath": "$context.resourcePath", 
    "requestHostHeader": "$context.domainName",
    "requestUserAgentHeader": "$context.identity.userAgent",
    "ip": "$context.identity.sourceIp", 
    "status": "$context.status",
    "responseLength":"$context.responseLength", 
    "durationMs": "$context.responseLatency",
    "caller": "$context.identity.caller", 
    "user": "$context.identity.user",
    "principalId": "$context.authorizer.principalId",
    "cognitoIdentityId": "$context.identity.cognitoIdentityId",
    "userArn": "$context.identity.userArn",
    "apiKey": "$context.identity.apiKey"
}

Click Save Changes and that’s it! API logs should start showing up after a few minutes.

Manual Firehose Creation

If you cannot use the CloudFormation template or having issues, you can manually create and configure a Kinesis Data Firehose using the HTTP destination.

Go to Amazon Kinesis within the AWS Console and select Create Delivery Stream.

The name of your delivery stream must start with amazon-apigateway-

  • For the source, select Direct PUT or other sources.
  • For the destination, select HTTP Endpoint and enter the following:
  • For HTTP endpoint URL, enter https://api.moesif.net/v1/partners/aws/kinesis
  • For Access key, enter your Moesif Application Id.
  • For Buffer interval, enter 60 seconds.
  • Enable GZIP compression.

Finish the creation wizard and make note of the ARN for Step 2

Updated: