Secure Proxy

With Secure Proxy, you can gain the privacy benefits of an on-premises installation without the cost of maintaining your own data infrastructure using client-side encryption. With Secure Proxy, you can deploy Moesif in a regulated environment with sensitive data like financial or healthcare and meet compliance requirements like HIPAA (view more on data security and compliance).

To view a case study on Secure Proxy, view How pVerify Deploys Moesif HIPAA-Compliant API Analytics for COVID Testing

How it works

The secure proxy is a Docker container deployed in your infrastructure which handles client-side encryption using Bring Your Own Key (BYOK) on the fly. Because Moesif doesn’t have access to your master encryption keys, its employees cannot access or view your API data.

How Moesif works with secure proxy

Keystore

The secure proxy has adapters for popular key stores to handle storing and rotating your master encryption keys. These keys are never transmitted or stored on Moesif servers. Currently, AWS Key Management Service (KMS) and AWS CloudHSM are supported. If you require an alternative, reach out our success team.

Even if you don’t deploy secure proxy, Moesif still encrypts your data at rest, in motion, and within our infrastructure using Moesif-managed keys. Secure proxy is an additional layer enabling you to encrypt using customer-managed keys.

Features supported

Almost all Moesif features will function normally even with the secure proxy deployed including our Management API, alerting, embedded dashboards, and more. Moesif built a complex analytics store which works on encrypted data which enables secure proxy to work. There are still some minor limitations which we recommend reviewing. If there are any questions,reach out our success team who can guide you on whether secure proxy will fit your requirements.

Access

There are two different ways to deploy secure proxy depending on whether you require the use of Moesif’s embedded dashboards feature:

  • Local network only: If you don’t plan to use Moesif’s embedded dashboards feature, you can deploy the secure proxy on an internal host that’s not exposed to the internet. However, the host must be accessible by an employee’s laptop when they log into the Moesif portal such as by being on a corporate VPN.

  • Exposed to the internet: If you plan on using Moesif’s embedded dashboards feature, the secure proxy must be accessible by the internet so that your customers can also decrypt their data on the fly. The recommended way to do so is to deploy NGINX with SSL in front of Moesif secure proxy with SSL. A Docker example with NGINX and Moesif Secure Proxy is available on GitHub. Once done, you can add a CNAME or A Record to your DNS settings like analytics.acmeinc.com. Employees can also access the Moesif portal on the go and do not need to be on a corporate VPN.

What’s encrypted

All sensitive data such your HTTP headers, request and response body, and event metadata is encrypted. There are non-sensitive elements such as request time that’s not encrypted for proper analytics. More info.

How to install

Secure proxy is available only on enterprise plans

1. Run the Docker image

A working docker-compose.yml is available on GitHub with moesifproxy and NGINX configured with SSL termination using Let’s Encrypt.

The easiest way to run the secure proxy is as a Docker container. You need to set your Moesif and AWS keys as Docker environment variables such as via the -e option.

Option 1: AWS KMS access with access keys

In this opion, pass the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, as env variables which will be used to connect to AWS KMS

docker run -it  -e MOESIF_MANAGEMENT_API_KEY="Your Moesif Management Key" \
 -e MOESIF_APPLICATION_ID="Your moesif application Id" \
 -e AWS_CUSTOMER_KEY_ID="Your customer managed key(CMK) in AWS KMS" \
 -e AWS_ACCESS_KEY_ID="Your AWS access Key Id" \
 -e AWS_SECRET_ACCESS_KEY="Your AWS secret access key" \
 -p 9500:9500 --name moesifproxy moesifproxy

Option 2: AWS KMS access assuming a IAM role

When Secure proxy is run in AWS, AWS supports multiple ways to inject the auth credentials directly to a EC2 instance or via Kubernetes service accounts. In this case pass the IAM role AWS_SECURE_PROXY_ROLE_ARN as environment variable, Secure Proxy will assume this role to gain auth access to AWS KMS. Note for Secure Proxy to assume role AWS_SECURE_PROXY_ROLE_ARN, base creds should have been injected already by AWS or K8s service accounts to pods.

docker run -it  -e MOESIF_MANAGEMENT_API_KEY="Your Moesif Management Key" \
 -e MOESIF_APPLICATION_ID="Your moesif application Id" \
 -e AWS_CUSTOMER_KEY_ID="Your customer managed key(CMK) in AWS KMS" \
 -e AWS_SECURE_PROXY_ROLE_ARN="IAM role that secure proxy should assume to acess AWS KMS" \
 -p 9500:9500 --name moesifproxy moesifproxy

Note:

  1. MOESIF_MANAGEMENT_API_KEY must have scopes create:encrypted_keys, read:encrypted_keys
  2. Optionally AWS_KMS_REGION can be passed as env variable. Defaults to us-west-2

2. Configure Moesif SDK

The collection API is accessible via the /collector endpoint on the secure proxy. You will need to configure any SDKs to log to this endpoint instead of Moesif’s main API at https://api.moesif.net.

For example, instead of logging to https://api.moesif.net/v1/events, you can log traffic to the the secure proxy via http://localhost:9500/collector/v1/events.

To do so, override the Moesif SDK Base URI to route through your local proxy hostname. Refer to your respective server integration docs or reach out to support.

An example for moesif-nodejs is below:

const moesifMiddleware = moesif({
  applicationId: process.env.MOESIF_APPLICATION_ID,
  baseUri: 'http://localhost:9500/collector',
});

Accessing the Moesif Management API

Like the collector API, you can access your data programmatically via the Moesif Management API while having it decrypted on the fly as long as you route your requests through the proxy under the /api endpoint. Keep in mind any scripts or access you make must have access to your local proxy instance.

For example instead of querying data in Moesif directly via https://api.moesif.com/v1/search/~/search/events, you can search data in Moesif like so:

curl -XPOST http://localhost:9500/api/search/~/search/events

Scaling Moesif proxy

For light load, you may be fine with a single Docker container running. If you expect a high volume of data collected or a large number of users at your organization querying data in Moesif, you can create a pool of Moesif proxy instances behind a load balancer. No storage is required which simplifies scaling. You can even run multiple clusters at different host names for better control. For example, one cluster can be used for data collection from SDKs, while a different cluster is used for users querying data in Moesif.

Configuration options

MOESIF_APPLICATION_ID

Required Your Moesif Collector Application Id which can be found by logging into Moesif and going to API Keys from the top-right menu.

MOESIF_MANAGEMENT_API_KEY

Required Your Moesif Management API key which can be found by logging into Moesif and going to API Keys from the top-right menu. Ensure the key is generated with at least the create:encrypted_keys, read:encrypted_keys scopes.

MOESIF_ENCRYPTION_ENABLED

Set to false to disable client-side encryption.

MOESIF_ENCRYPT_NUMERICS

Set to false if numbers are not sensitive and don’t require encryption. This enables numeric math in Moesif.

MOESIF_ENCRYPT_DATES

Set to false if dates are not sensitive and don’t require encryption. This enables date math in Moesif.

MOESIF_ENCRYPT_REQUEST_HEADERS

Set to false if request headers are not sensitive and don’t require encryption.

MOESIF_ENCRYPT_RESPONSE_HEADERS

Set to false if response headers are not sensitive and don’t require encryption.

MOESIF_ENCRYPT_METADATA

Set to false if metadata is not sensitive and don’t require encryption.

MOESIF_ENCRYPT_GRAPHQL_QUERY

Set to true if GraphQL query string is sensitive and requires encryption (uncommon). Moesif will encrypt the GraphQL query in request body. When encrypted, GraphQL analytics will have limited functionality. Default to false, where Moesif will encrypt all the fields in the request body except query field.

MOESIF_ENCRYPT_REQUEST_HEADERS_BLACKLIST

Set a string(comma separated) which can be a blacklist of fields that will bypass request headers encryption. For example - If MOESIF_ENCRYPT_REQUEST_HEADERS_BLACKLIST=Host is set, then Moesif will skip encrypting host at any level while creating an event. Please note that this configuration option is case insensitive.

MOESIF_ENCRYPT_RESPONSE_HEADERS_BLACKLIST

Set a string(comma separated) which can be a blacklist of fields that will bypass response headers encryption. For example - If MOESIF_ENCRYPT_RESPONSE_HEADERS_BLACKLIST=x-moesif-transaction-id is set, then Moesif will skip encrypting x-moesif-transaction-id at any level while creating an event. Please note that this configuration option is case insensitive.

MOESIF_ENCRYPT_METADATA_BLACKLIST

Set a string(comma separated) which can be a blacklist of fields that will bypass metadata encryption. For example - If MOESIF_ENCRYPT_METADATA_BLACKLIST=datacenter,sf is set, then Moesif will skip encrypting datacenter and sf at any level while creating an event. Please note that this configuration option is case insensitive.

AWS configuration config

The below options are used to configure AWS KMS.

AWS_CUSTOMER_KEY_ID

AWS_CUSTOMER_KEY_ID is the Customer managed key (CMK) in AWS KMS, To find KeyId follow the instructions here

AWS_KMS_REGION

AWS_KMS_REGION is a string representing aws region where AWS KMS is configured. Defaults to ‘us-west-2’

AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY

These are access keys needed to access AWS KMS via api. More information about access keys here. Make sure that IAM user/role associated with the access keys has permissions to access AWS KMS.

AWS_SECURE_PROXY_ROLE_ARN

This is the ROLE ARN that secure proxy should assume to access AWS KMS

How to build

Moesif Proxy uses the revel webframework. To learn more about revel https://revel.github.io/

To build Docker image

docker build -t moesif/moesifproxy:latest

To push to Docker repo

docker login

docker push moesif/moesifproxy:latest

Limitations

  • If enabled for a pre-existing app, filtering and aggregations will not work on old data. We recommend creating a new app in Moesif for enabling secure proxy.
  • Numeric fields will behave like strings (i.e. no math supported) unless MOESIF_ENCRYPT_NUMERICS is set to false.
  • Date fields will behave like strings (i.e. no date math supported) unless MOESIF_ENCRYPT_DATES is set to false.
  • If using embedded templates (for customer-facing portal), the secure proxy must be accessible from the internet.
  • We recommend a subdomain like analytics.acmeinc.com that points to NGINX with SSL (See Docker example).

Unencrypted data

There are certain parts of your API logs that bypass client-side encryption for proper analytics, including:

  • Request and response time (latency)
  • Route and verb
  • Response status code
  • user_id and company_id
  • Binary and non-JSON payloads such as Protobuf or Thrift
  • The schema/names of fields used within your API

Like most web analytics tools, the secure proxy does not encrypt data from client integrations like moesif-browser-js, which is used for tracking user attributes and actions like “Signed In” or “Purchased Plan”.

Other use cases

Besides client-side encryption and decryption, the secure proxy enables a variety of other use cases:

  • Reduce the number of outbound connections to Moesif’s APIs.
  • Relay requests from servers that don’t have direct access to the internet.
  • White label the Moesif Collector or Management API like analytics.acmeinc.com/api.

Updated: