Security Best Practices for Amazon Elasticsearch - Part One

According to Duo in 2018, there were “16K public IPs of exposed AWS managed ElasticSearch [sic] clusters that could have their contents stolen or possibly data deleted.”

There have been many reports of data exfiltration and malicious data deletion due to publicly exposed Elasticsearch clusters in recent years. The need for standardized best practices for Elasticsearch is paramount for organizations of all sizes to avoid these risks. Currently there are many established best practices and frameworks designed to guide organizations to become more secure such as the Center for Internet Security (CIS) and National Institute of Standards and Technology (NIST), but these standards do not provide detailed guidance for Elasticsearch clusters.

At ScaleSec, we partner with Small And Midsize Businesses (SMB), and large organizations to secure their cloud environments and frequently see a need for established security best practices for specific AWS services. Today we are releasing Part 1 of our internal security best practices for Amazon Elasticsearch. We hope this will assist you in further securing your cloud environment and protecting your data. Part 2 will be released in the coming weeks and will wrap up our ES security best practice recommendations.

What is Amazon Elasticsearch?

Amazon Elasticsearch (Amazon ES) is a fully managed AWS service that lowers the barrier of entry to deploy, manage, and secure an Elasticsearch cluster in the cloud. Elasticsearch is an open source search and analytics engine that can process a varying number of different data types. It is commonly integrated into an ELK stack or EKK stack. Amazon bundles the Open Distro for Elasticsearch plus a built-in Kibana plugin for their managed offering in AWS along with numerous other integrations with AWS services.

Note: An Amazon ES “domain” and Amazon ES “cluster” are synonymous and will be used interchangeably throughout the article.

Security Best Practices

AWS Elasticsearch provides many different configuration options. At first glance such variety can be a bit overwhelming. Our recommendations aim to minimize the complexity of configuring and deploying an ES cluster while maximizing usability and security. The guidance detailed in this blog is based on industry standard security best practices as well as our experiences with our customers. Many organizations have different compliance or regulatory requirements, security threat levels, or leverage Amazon Elasticsearch in different ways. Due to the varying degree of Elasticsearch usage, some of the security recommendations might not directly align with your business needs. For example, organizations might enable Multi-factor Authentication (MFA) on their Single sign-on (SSO) provider instead of on Amazon Cognito. Either configuration of MFA would fulfill our recommendation of using MFA for authentication purposes.

Security topics covered in Part 1 include:

• Infrastructure
• IAM

Infrastructure Recommendations

Deploy Amazon Elasticsearch Into Its Own VPC

When deciding where to deploy an Elasticsearch domain in AWS there are two options available: VPC access or Public access. VPC access is recommended, because by default it provides secure communication between the ES domain and other services within the VPC. Once an ES cluster has been deployed to a VPC, only users with access to that VPC can reach the cluster. Requests originating from the public internet do not reach the cluster and are never analyzed for authentication or authorization. By blocking public requests you are lowering your attack surface by stopping potential threats before they hit your resources. This configuration provides the first layer of security that will be detailed in this article.

Note: Once you select VPC access you cannot change the domain to Public access and vice versa. You must create a new domain and migrate your data.

Leverage Security Groups for IP-Based Access Control

When an ES cluster is deployed in a VPC, each data node receives an Elastic Network Interface (ENI). Each ENI receives a private IP address in addition to a DNS hostname that ties the two together. In order for a user to interact with the ES domain hostname for the cluster, the security group attached to the ENI must allow that traffic. Security groups are virtual stateful firewalls that control inbound and outbound traffic using IP and port-based rulesets. It is recommended to only add rules to the ES security groups that are scoped to the IPs required to communicate with the ES domain. Avoid large or generic CIDR ranges in your security groups’ rulesets to avoid accidental exposure to unauthorized users or applications. Security groups are the second line of defense for Amazon Elasticsearch clusters after VPC access.

IAM Recommendations

Enable Fine-grained Access Control

Fine-grained access control for Amazon ES utilizes the Open Distro for Elasticsearch which contains security features that together encompass the fine-grained access control functionality you see in Amazon ES. When fine-grained access control is enabled the following features are default requirements that you cannot turn off:

• node-to-node encryption
• encryption at rest
• HTTPS / TLS Network Communication

In addition, least privileged access can be configured by leveraging authentication via Amazon Cognito for Kibana. Amazon Cognito provides flexible configuration options for Kibana such as roles, SAML federation via external Identity Providers (IdP), MFA, and much more. We recommend using Amazon Cognito for Kibana as your authentication service in addition to enabling fine-grained access control for Amazon Elasticsearch.

One of the big advantages of fine-grained access control is its ability to provide security at the index, document, and field level inside ES clusters. This is configured with Kibana roles, which are different from IAM roles. Kibana roles can be scoped to one or many different permission levels including cluster-wide, index-specific, document, and field levels. These roles are then mapped to users or applications which form the authentication and authorization for requests to the ES domain.

Reference: https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/fgac.html

Utilize Domain Access Policies

Domain access policies are resource-based policies for Amazon Elasticsearch clusters that detail which users or applications can perform specific actions on a domain’s subresources. In Amazon ES, a subresource can be an Elasticsearch index or API. When a request is allowed by a security group it reaches a domain endpoint and is evaluated by the domain access policy. The access policy will either allow or deny the request access to the ES cluster itself based on the policy configuration. All of this happens before the request makes it to the Elasticsearch cluster. This means that untrusted sources are filtered out before the request interacts with the cluster.

There are three options available to you for Domain access policies:

• Custom access policy
  • IPv4 address
    • This type of custom access policy is not available with VPC access enabled. Instead, use security groups for IP-based access controls.
  • IAM ARN
    • This type of custom access policy accepts an AWS account ID, account ARN, IAM user ARN, or IAM role ARN and applies either an ALLOW or DENY access level to the ES domain’s subresources. Translated to a JSON policy we see that a wildcard es:* is configured in addition to a wildcard in the resource block to allow access to all of the domain’s subresources:

• • Wildcard (*) IAM ARN Custom access policies give the principal(s) specified FULL ACCESS to the ES domain’s subresources. A domain access policy with a principal of the account ID or account ARN suffices if Fine-Grained Access Control is enabled because you are offloading the authentication and authorization to Amazon Cognito.
  • Diving into the specifics of IAM and access control for Amazon Elasticsearch will be discussed in a future article, but we generally see the most success with a broader domain access policy and leveraging fine-grained access control with Amazon Cognito for authentication.
• JSON defined access policy
  • JSON defined access policies allow the most control and configuration over who or what can access the ES cluster. We recommend this type of domain access policy in conjunction with Identity-based policies if Fine-grained access control is not enabled.
• Allow open access to the domain (Not Recommended)
  • This access policy removes this layer of request evaluation and allows all requests to reach the ES cluster for further authentication and authorization.

Properly Scope Identity-based Policies

Identity-based policies have the ability to provide access to an Amazon ES domain’s configuration in addition to its subresources. It is critical to properly scope IAM and be aware of the impacts an overly-permissive IAM policy might have if compromised. Avoid providing es:* permissions with a resource of ”*” because this allows anyone with that IAM policy to delete or update the entire ES domain.

Configure Index, Document, and Field-Level Security

A feature of the Open Distro for Elasticsearch (Fine-grained access control) is the ability to restrict indexes, documents, and fields to specific users and Kibana roles. We recommend following least privilege access when it comes to who or what can access data in Elasticsearch clusters.

Field masking is another alternative to field-level security. When this is configured it will return the field but mask the field value. Index, document, and field-level security will remove the restricted content in the response from Elasticsearch. Generally we see a custom approach to restricting specific indices, documents, or fields depending on the organization’s use cases.

Configure MFA on Amazon Cognito User Pools

Multi-Factor Authentication (MFA) provides a second form of authentication in addition to a username and password. Cognito user pools support SMS text messages or time-based one-time passwords (recommended) for MFA. There are different settings available for MFA including Off, Optional, or Required. We recommend Required if you are not using Advanced Security which has the ability to perform risk-based analysis to prompt for MFA.

Note: Amazon Cognito also supports SAML federation using an external Identity Provider (IdP). This scenario may better fit your Organization and configuring MFA on the IdP is preferred for that configuration versus configuring on the user pool itself.

Set IAM ARN as Master User

After you enable fine-grained access control you have two options for a master user: IAM ARN or Create master user. We recommend an IAM ARN as your master user because it requires all requests to the cluster to be signed with AWS Signature Version 4. If you were to select Create master user, you would not be able to use the same internal user database between different ES Domains. You can read more about the master user and its usage in Amazon Elasticsearch here.

Conclusion

Throughout this article we reviewed and recommended multiple security best practices for Amazon Elasticsearch related to infrastructure and IAM. Not all of the recommendations listed will pertain to your environment or specific use case of Amazon Elasticsearch. At the bare minimum, we highly recommend deploying your Amazon Elasticsearch Domains into a VPC and enabling fine-grained access control.

We recommend reading Best Practices Part Two where we cover encryption, logging and monitoring, and general configuration.

Thanks to @Dustin Whited, @Sarah Gori, and John Porter for their editor contributions.