Encryption in The Cloud Pt. 1

A multipart exploration into Cloud Encryption. Part 1: Encryption in Amazon S3

Introduction

This is the first of a multipart dive into encryption in the public cloud. Part 1 will focus on encryption in Amazon S3. Part 2 will focus on encryption in GCP’s Google Cloud Storage (GCS). Unless specified, the options mentioned herein use AES-256 encryption. Topics that will be covered for Amazon S3:

Encryption at Rest
Server-Side Encryption
Client-Side Encryption
S3 Default Encryption
Envelope Encryption

See Appendix A for acronyms and keys for cloud terminologies.

1. Encryption at Rest

Encryption at rest refers to encrypting data or objects when in storage or “at rest”. Encrypting data at rest safeguards your data at the object level when it writes to S3 storage. Two ways to achieve Encryption at Rest are server-side and client-side encryption.

2. Server-Side Encryption

With Server-Side Encryption (SSE), your data is encrypted AFTER you send it to the cloud. If you decide to utilize SSE, it is always recommended to transfer data using TLS protected communications. There are three options for SSE in S3. In order from least control to most control for the end user:

Server-Side Encryption with Amazon S3-Managed Keys (SSE-S3)
Server-Side Encryption with AWS KMS-Managed Keys (SSE-KMS)
Server-Side Encryption with Customer-Provided Keys (SSE-C)

2.1. Server-Side Encryption with Amazon S3-Managed Keys (SSE-S3)

The SSE-S3 option uses specialized AWS owned CMKs that end users cannot interact with through the console or API. AWS owned CMKs are not stored in the end user’s account and can only be called by AWS Services on-behalf of the user. End users cannot view, manage, or audit AWS owned CMKs because AWS owns and manages the keys for you. Users do not need permissions for the keys to view objects — only authorization to the objects themselves, this is commonly referred to as Transparent Data Encryption. This encryption option may not be suitable for production environments due to compliance requirements for key management.

2.2. Server-Side Encryption with AWS KMS-Managed Keys (SSE-KMS)

The SSE-KMS option leverages the AWS KMS service and requires the end user to have permissions for the key used AND the object requested. If the user does not have both permissions, the request will fail. SSE-KMS keys are stored in AWS KMS in the user’s AWS account.

S3 can use two types of CMKs with KMS— Customer-managed CMKs or AWS-managed CMKs.

2.2.1 Customer-managed CMKs

Customer-managed CMKs are keys that the end user creates, owns, manages and interacts with directly.

AWS does not rotate these keys by default and places the responsibility for access management and overall ownership on the end user.
Customer-managed CMKs incur a monthly fee (in excess of the free tier) and count against AWS KMS resource limits per account.
This encryption type is recommended if you plan on sharing objects across accounts due to your ability to modify the key policy.

2.2.2 AWS-managed CMKs

AWS-managed CMKs can only be requested by AWS Services but are stored in the end user’s AWS account.

AWS-managed CMKs are auditable with CloudTrail and have a viewable key policy (cli only) but you cannot edit the key policy.
AWS manages the key rotations on your behalf for all “AWS-managed” keys.
In the console, AWS-managed CMKs are identifiable by their unique alias format. aws/service-name such as aws/s3. This format is reserved and users are not able to use it for their Customer-managed CMK aliases.

2.3. Server-Side Encryption with Customer-Provided Keys (SSE-C)

The SSE-C option offloads key management to the end user in lieu of AWS KMS. Instead of services or end users interacting with keys inside KMS, the encryption key is provided as part of the operation request to S3. AWS still manages the encryption / decryption of the object so no additional code is needed to perform those operations. Each object and each version of the object can have a different encryption key. The user is required to track which key is used for each object and supply that during the request.

Important: Amazon S3 does not store the encryption key you provide. Instead, it stores a randomly salted HMAC value of the encryption key to validate future requests. The salted HMAC value cannot be used to derive the value of the encryption key or to decrypt the contents of the encrypted object. That means if you lose the encryption key, you lose the ability to decrypt the object. Server-side encryption encrypts only the object data, not object metadata.

Reference: https://docs.aws.amazon.com/AmazonS3/latest/dev/ServerSideEncryptionCustomerKeys.html

Comparison of the Amazon S3 Encryption Options

3. Client-Side Encryption

With Client-Side Encryption, your data is encrypted BEFORE you send it to S3. Client-side Master Key and AWS KMS–Managed Customer Master Key (CMK) are AWS offerings for Client-Side Encryption. Both offerings utilize symmetric keys for encryption of data called Data Encryption Keys (DEKs). DEKs are typically either used in memory on the KMS HSMs or returned to the customer for operations on the client side.

Client-Side Encryption operations are performed by customer applications using one of the AWS SDKs that supports client-side encryption.

3.1 AWS KMS–Managed Customer Master Key (CMK)

For each request to S3 you reference a CMK ID which reaches out to KMS to generate a DEK. This additional request happens before the object is sent to S3.

For each upload request to S3, KMS returns two values:

A plaintext version of the encryption key (DEK) to encrypt the data
A cipher blob of the same key that is then uploaded as object metadata

After the values are returned, the plaintext DEK and object are sent to the local encryption client for encryption and are then uploaded. The plaintext DEK is then wiped from memory.

For each download request:

The CMK cipher blob is returned with the object
The cipher blob is sent to KMS to get the plaintext version of the DEK to decrypt the data

Note: The plaintext key is sensitive and should be treated as such.

Reference: https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/how-it-works.html

3.2 Client-side Master Key

Client-side Master Key encryption has the same restrictions and similar functionality of SSE-C but it happens on the client side. An example of the AWS SDK for Java encrypt/decrypt flow is documented below and a pre-requisite is to supply a client-side master key to the local encryption client.

Encryption Steps:

Generate a one time use symmetric key (referred to as DEK).
Encrypt the file data using this DEK.
Encrypt the DEK using the client-side master key.
Store the encrypted DEK with the encrypted file.
Store a description of the client-side master key alongside the DEK to uniquely identify the key used to encrypt the DEK.

Decryption Steps:

Retrieve the encrypted DEK you stored with the encrypted file.
Retrieve the description of the original client-side master key.
If the description of the master key on hand does not match the description of the original master key, use the unique description to fetch the original master key used to encrypt the DEK.
Decrypt the DEK using the master key.
Decrypt the file data using the plaintext DEK.

More detailed information can be found here: https://aws.amazon.com/articles/client-side-data-encryption-with-the-aws-sdk-for-java-and-amazon-s3/

4. Default Encryption

Amazon S3 Default Encryption is a feature offering from AWS that allows you to set a default encryption behavior for a S3 bucket. This features allows you to encrypt objects without specifying an encryption option each time you upload an object. This feature is offered with no charge, but you may incur fees depending on the key type used.

There are currently two available selections for the Default Encryption feature: SSE-S3 and SSE-KMS. You must turn on default encryption to utilize this feature but it also can be added after bucket creation. If default encryption is enabled after the bucket is created and objects have already been uploaded, those objects ARE NOT encrypted automatically.

5. Envelope Encryption

Envelope encryption is the concept of encrypting keys with keys. The data key encrypts your data and then that data key is encrypted with a different key, called the customer master key. The envelope process repeats itself and eventually ends with one heavily protected and distributed plaintext key. Occasionally you may hear encrypted data keys referred to as “wrapped”. This signifies envelope encryption was used.

5.1 Data Keys and S3 Objects

When uploading an object to S3, an encrypted copy of the DEK is stored alongside the object for low latency operations when a request is made. Amazon S3 is designed for “11 nines” of durability (99.999999999%) for objects stored on standard-tier S3. In pursuit of this design goal, AWS redundantly stores your objects across many different devices (storage arrays) in a single region with no action needed on your end.

What does this mean for the DEK? Well, the DEK is also stored on multiple storage arrays but S3 does NOT copy the DEK as a whole object. What ends up happening is the DEK and the object are chunked together and those paired chunks are stored across different storage arrays (sometimes multiple on the same array, but enough for the durability goal). An example flow is shown below to visualize the DEK and S3 Object chunk and storage process.

Achievement Unlocked: Another Cloud Security Competency Realized

5.2 Key Hierarchy

A brief breakdown of the KMS envelope key hierarchy will follow but first we need to touch-base on the concept of a Domain in regards to KMS.

5.2.1 Domains

A domain is a regionally defined set of AWS KMS servers, HSMs, and operators. There are shared keys in the domain called domain keys. All of the HSMs inside the domain share a synchronized state, called a domain state. The domain state is exported as a domain token.

Domain — Collection of HSMs backing the AWS KMS service in a region
Domain Keys — Secret Keys shared across the domain
Domain State — A synchronized state shared between the HSMs in a domain
Domain Token — The Domain State is synchronized across the domain via this exported token

5.2.2 KMS Key Hierarchy

Domain keys are at the top of the key hierarchy. Domain keys wrap CMKs and CMKs generate and wrap the data keys.

6. Summary and Next Installment

To summarize, we covered the encryption options in Amazon S3 ranging from client and server-side encryption, the specific key methodologies available to you through KMS, key hierarchies and envelope encryption. Part 2 of this series will focus on the Google Cloud Storage service and the encryption options available to you. For a deeper dive into any of the topics covered, reference Appendix B. If there’s a topic or area you’d like us to cover, ping us on all social.

Appendix A — Abbreviations and Keys

This section contains abbreviations and keys.

AWS — Amazon Web Services
KMS — Key Management Service
AES — Advanced Encryption Standard
CMK — Customer Master Key
DEK — Data Encryption Key
HMAC — Hash-based Message Authentication Code
HSM — Hardware Security Module
KEK — Key Encryption Key
S3 — Simple Storage Service
SDK — Software Development Kit
SSE-C — Server-Side Encryption with Customer-Provided Keys
SSE-S3 — Server-Side Encryption, Simple Storage Service
SSE-KMS — Server-Side Encryption, Key Management Service
TLS — Transport Layer Security