Encrypting data on S3

In this recipe, we will learn to encrypt data on S3 at rest using server-side encryption techniques. Encryption on the server side can be done in three ways: server-side encryption with S3-managed keys (SSE-S3), server-side encryption with KMS-managed keys (SSE-KMS), and server-side encryption with customer-provided keys (SSE-C). In client-side encryption, data is encrypted on the client side and then sent to the server.

Getting ready

We need a working AWS account with the following resources configured:

1. A bucket: I will be using a bucket with the name awsseccookbook. Replace it with your bucket name.
2. A user with administrator permission on S3: Configure a CLI profile for this user. I will be calling both the user and the profile on the awssecadmin CLI.
3. A customer-managed key created in KMS: Follow the Creating keys in KMS recipe in Chapter 4, Key Management with KMS and CloudHSM, to create a key. I have created one named MyS3Key.

How to do it...

In this recipe, we will learn about various use cases for server-side encryption.

Server-side encryption with S3-managed keys (SSE-S3)

We can upload an object from the console with SSE-S3 as follows:

1. Go to the S3 bucket.
2. Click Upload, click Add Files, select your file, and then click Next, selecting the defaults in the Set Properties tab.
3. In the Set Properties tab, scroll down and select Amazon S3 master key under Encryption. Follow the on-screen options to complete the upload. We can verify this from the object's properties:

It is important to note that, if we try to open or download the object, we will still be able to see the object as-is because S3 will decrypt the object using the same key.

We can change encryption for an existing object to SSE-S3 as follows:

1. Go to the object's Properties tab.
2. Go to Encryption, select AES-256, and then click Save:

We can upload an object from the CLI with SSE-S3 using the following command:

aws s3 cp image-heartin-k.png s3://awsseccookbook/image-heartin-k.png \
--sse AES256 \
--profile awssecadmin

Next, we will execute SSE with KMS managed keys.

Server-side encryption with KMS-managed keys (SSE-KMS)

We can upload an object from the console with SSE-KMS as follows:

1. Go to the bucket.
2. Click Upload, click Add Files, select your file, and then click Next, selecting the defaults in the Set Properties tab.
3. In the Set Properties tab, scroll down, select AWS KMS master key, and then select our KMS key (refer to the Getting ready section). Follow the options on the screen to complete the upload:

We can change encryption for an existing object to SSE-KMS as follows:

1. Go to the object's Properties tab.
2. Go to Encryption, select AWS-KMS, then select your KMS key (refer to the Getting ready section), and then click Save:

We can upload an object from the CLI with SSE-KMS using the following command:

aws s3 cp image-heartin-k.png s3://awsseccookbook/image-heartin-k.png \
--sse aws:kms \
--sse-kms-key-id cd6b3dff-cfe1-45c2-b4f8-b3555d5086df \
--profile awssecadmin

sse-kms-key-id is the ID of the KMS key you created (refer to the Getting ready section).

Server-side encryption with customer-managed keys (SSE-C)

We can upload an object from the CLI with SSE-C as follows:

1. Upload an object from the CLI with SSE-C by using the following command:

aws s3 cp image-heartin-k.png s3://awsseccookbook/image-heartin-k.png \
--sse-c AES256 \
--sse-c-key 12345678901234567890123456789012 \
--profile awssecadmin

2. Retrieve the object encrypted using SSE-C, providing the same key we used in the previous command, as follows:

aws s3 cp s3://awsseccookbook/image-heartin-k.png image-heartin-k1.png \
--sse-c AES256 \
--sse-c-key 12345678901234567890123456789012 \
--profile awssecadmin

If we do not specify the sse-c option while downloading an object encrypted with SSE-C, we will get an exception as follows: fatal error: An error occurred (400) when calling the HeadObject operation: Bad Request. If we do not specify the correct key that was used for encryption (using the sse-c-key option) while downloading an object encrypted with SSE-C, we will get an exception as follows: fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden.

How it works...

In the Server-side encryption with S3-managed keys (SSE-S3) section, we uploaded an object from the console with SSE-S3 encryption. We changed the encryption for an existing object to SSE-S3 encryption. We also uploaded an object with SSE-S3 encryption. When performing SSE-S3 encryption from the CLI, the value of the sse parameter is optional. The default is AES256.

In the Server-side encryption with KMS-managed keys (SSE-KMS) section, we uploaded an object with SSE-KMS encryption. We changed encryption for an existing object to SSE-KMS encryption. We also uploaded an object from the CLI with SSE-KMS encryption. When performing SSE-KMS encryption from the CLI, the value of the sse-c parameter is optional. The default is AES256.

In the Server-side encryption with customer-managed keys (SSE-C) section, we uploaded an object from the CLI with SSE-C encryption. Unlike the other two server-side encryption techniques SSE-S3 and SSE-KMS, the console does not currently have an explicit option for SSE-C. We will need to execute this using APIs. In this recipe, we used a 32-digit number as the key. However, in the real world, keys are generally generated using a key generation tool. We will learn more about keys when we discuss KMS later in this book.

There's more...

Let's quickly go through some important concepts related to S3 encryption:

• Data on S3 can be encrypted while at rest (stored on AWS disks) or in transit (moving to and from S3). Encryption at rest can be done using server-side encryption or by uploading encrypted data from the client.
• S3 server-side encryption techniques for data at rest use symmetric keys for encryption.
• Encryption of data in transit using SSL/TLS (HTTPS) uses asymmetric keys for encryption.
• S3 default encryption (available as bucket properties) provides a way to set the default encryption behavior for an S3 bucket with SSE-S3 or SSE-KMS. Enabling this property does not affect existing objects in our bucket, and applies only for new objects uploaded.
• With client-side encryption, we need to manage keys on our own. We can also use KMS to manage keys through SDKs. However, it is not currently supported by all SDKs.
• Encryption in transit can be achieved with client-side encryption or by using SSL/TLS (HTTPS).
• Server-side encryption types, SSE-S3 and SSE-KMS, follow envelope encryption, while SSE-C does not use envelope encryption.
• Some important features of SSE-S3 include the following:
  • AWS takes care of all key management.
  • It follows envelope encryption.
  • It uses symmetric keys to encrypt data.
  • Each object is encrypted with a unique key.
  • It uses the AES-256 algorithm.
  • A data key is encrypted with a master key that is automatically rotated periodically.
  • It is free.
• Some important features of SSE-KMS include the following:
  • Keys are managed by AWS KMS.
  • Keys can be shared by multiple services (including S3).
  • As customers, we get more control over keys, such as creating master and data keys, and disabling and rotating master keys.
  • It follows envelope encryption.
  • It uses symmetric keys to encrypt data.
  • A data key is encrypted with a master key.
  • It uses the AES-256 algorithm.
  • We can choose which object key to encrypt while uploading objects.
  • We can use CloudTrail to monitor KMS API calls, enabling better auditing.
  • It is not free.
• Some important features of SSE-C include the following:
  • Keys are managed by us (customers).
  • The customer provides a key along with data. S3 uses this key for encryption and deletes the key.
  • The key must be supplied for decryption as well.
  • It does not use envelope encryption.
  • It uses symmetric keys to encrypt data.
  • It uses the AES-256 algorithm.
  • AWS will force you to use HTTPS while uploading data since you are uploading your symmetric key as well.
  • It is free.
• By default, S3 allows both HTTP and HTTPS access to data. HTTPS can be enforced with the help of a bucket policy with the following condition element:

"Condition": {
    "Bool": {
        "aws:SecureTransport": "false"
    }
}

Any requests without HTTPS will fail with this condition.

See also

• You can read more about encryption at https://heartin.tech/en/blog-entry/important-points-remember-about-encryption.