In today's world of hyperconnectivity, keeping end-users, employees, company information, and devices safe on the internet can be a monumental task. With internet security on the mind of everyone - users demand privacy, employees want sensitive data stored safely, and management teams must to protect intellectual property - development teams must continuously thwart adversaries by design; anticipating, preventing, detecting, disrupting and containing their actions.

A popular technique to secure both communication in-transit and at-rest is encryption. Encryption has been keeping the modern internet safer for decades now. In the context of Google Cloud Platform, encryption is used to protect users and server communication (in-transit), protect data stored (at-rest), and can be extended to meet the needs of companies, developers, and end-users.

In this codelab, we will explore Encryption within Google Cloud Platform (GCP), from fundamentals to more advanced customizations. We'll take a look at the default behavior of GCP, learn about encryption within the developer environment, and deploy a sample Bookshelf app interacts with the Cloud Key Management Service (KMS) to encrypt application data.

What are we going to be building?

In this codelab, you are going to extend the existing Bookshelf web application and deploy encryption techniques in the application:

What you'll learn

What you'll need

This codelab is focused on encryption within GCP and Cloud KMS. Non-relevant concepts and code blocks are glossed over and are provided for you to simply copy and paste.

Self-paced environment setup

Create a new Project

If you don't already have a Google Account (Gmail or Google Apps), you must create one. Sign-in to Google Cloud Platform console (console.cloud.google.com) and create a new project:

Note that the Project name "My Encryption Project" has already been taken and will not work for you, sorry! You will supply your own unique Project name. Remember this project ID; it is a unique name across all Google Cloud projects. It will be referred to later in this codelab as PROJECT_ID.

Next, you'll need to enable billing in the Cloud Console in order to use Google Cloud resources. New users of Google Cloud Platform are eligible for a $300 free trial.

To interact with the Google Cloud Platform we will use the Google Cloud Shell throughout this code lab.

Google Cloud Shell is a Debian-based virtual machine pre-loaded with all the development tools you'll need that can be automatically provisioned from the Cloud Console. This means that all you will need for this codelab is a browser (yes, it works on a Chromebook).

To activate Google Cloud Shell, from the Cloud console simply click the button on the top right-hand side (it should only take a few moments to provision and connect to the environment):

Once connected to the cloud shell, you should see that you are already authenticated and that the project is already set to your PROJECT_ID. Run the following command and you should see the following output:

gcloud auth list

Command output

Credentialed accounts:
 - <myaccount>@<mydomain>.com (active)
gcloud config list project

Command output

[core]
project = <PROJECT_ID>

If for some reason the project is not set, simply issue the following command :

gcloud config set project <PROJECT_ID>

Looking for you PROJECT_ID? It's the ID you used in the setup steps. You can find it in the console dashboard any time:

In this section, we'll download the sourcecode for the existing Bookshelf App for Java, configure Google Cloud Storage, and prepare the Google Cloud Shell environment to run the app.

Enable the Datastore API

The application will use Google Cloud Datastore to store application data. Go to the Datastore API dashboard, and enable the API.

Switch to Java 1.8

Within the Google Cloud Shell, the default version of the Java environment is 1.7. This example requires version 1.8 of the JRE and JDK. Enter the following at the Cloud Shell command prompt:

$ sudo update-java-alternatives -s java-1.8.0-openjdk-amd64

Now, test the version of Java that is configured to verify it has been switched from 1.7 to 1.8.

$ java -version

Command output

openjdk version "1.8.0_111"
OpenJDK Runtime Environment (build 1.8.0_111-8u111-b14-2~bpo8+1-b14)
OpenJDK 64-Bit Server VM (build 25.111-b14, mixed mode)

The Apache Maven client requires that the JAVA_HOME and path to the Java compiler be set appropriately. Execute these two commands to set the variables. If the window closes, these will have to be reset.

$ export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64
$ export PATH=$PATH:$JAVA_HOME/bin/

Clone repository

Within the Google Cloud Shell, use git to make a copy and download the sample project locally. Copy and paste this git clone command directly into the Shell and execute.

$ git clone https://github.com/GoogleCloudPlatform/getting-started-java.git

Command output

Cloning into 'getting-started-java'...
remote: Counting objects: 3401, done.
remote: Compressing objects: 100% (27/27), done.
remote: Total 3401 (delta 13), reused 0 (delta 0), pack-reused 3374
Receiving objects: 100% (3401/3401), 50.08 MiB | 5.88 MiB/s, done.
Resolving deltas: 100% (1754/1754), done.
Checking connectivity... done.

Alternatively, you can download the sample as a zip file and extract it.

Now, change the current directory to ./getting-started-java/bookshelf/3-binary-data/ folder.

$ cd ./getting-started-java/bookshelf/3-binary-data/

Create storage bucket

Now we need to make some minor configuration changes to the sample application, setting the Google Cloud Storage (GCS) bucket you want the application to read-from/write-to.

First, let's create a new GCS bucket to read-from/write-to Bookshelf thumbnail images requested or uploaded by users within the Bookshelf sample application. We'll use this <new-unique-bucket-name> in the app configuration step to follow. It is ok for the bucket to be named the same as your project.

$ gsutil mb gs://<new-unique-bucket-name>

Command output

Creating gs://<new-unique-bucket-name>/...

Now go back to the Google Cloud Shell and run the following command to build and run the Bookshelf application locally. In this case, the application will make use of Google's default encryption at-rest.

Run the Bookshelf app:

$ mvn -Plocal clean jetty:run-exploded -DprojectID=[YOUR-PROJECT-ID] -Dbookshelf.bucket=[YOUR-GCS-BUCKET-NAME]

The Apache Maven client will download dependencies, build the source code, and launch a local version of the Bookshelf app.

Command output

[INFO] Building bookshelf-3 1.0-SNAPSHOT
...
[INFO] Started Jetty Server
...

Launch the web application in a browser:

Once the local server has started, it is time to preview the running application in the web browser. Click on the web preview button within the Google Cloud Shell UI. Select Preview on port 8080, the default.

A new tab will be owned with the running Bookshelf sample application. The Google Compute Engine or Google App Engine app is running locally on the Google Cloud Shell machine, which is, by default, accessible using the web Preview functionality.

Create a new Bookshelf book, define the name, select an image, and save it.

Optional: Create a new Bookshelf book. Understand how data is stored in the default behavior.

  1. Open Google Cloud Storage browser in the Cloud Console and inspect uploaded/stored images.
  2. Is the image encrypted using customer-supplied encryption? Open the metadata.
  3. Where is the metadata about the book (e.g. title, author) stored?

Now, let's configure Google Cloud Key Management Service (KMS) and prepare it as a service that can be integrated into the Bookshelf app. We'll use Cloud KMS to manage encryption keys and the process of encrypting/decrypting those keys.

Enable Cloud KMS

By default, Cloud KMS is not enabled. Visit the Cloud Console in your project and enable it. You must accomplish two steps to enable Cloud KMS.

  1. Billing account must be enabled (you should have completed this in a previous step)
  2. The Cloud KMS API must be turned on
    Cloud Console > API Manager > Google Cloud KMS API > Enable

Create a KeyRing

Now that the Cloud KMS API is enabled, it is time to create your first KeyRing. Navigate to the IAM and Admin menu, then click on Cloud KMS.

Then click the Create key ring button.

Give the key ring a name of testkeyring and set the location of this key ring to global. You could also restrict the availability of this key ring to a specific Google Cloud Platform Region.

Click the Create button.

Optional: You may also create a keyring using the command-line interface (gcloud) or Cloud KMS API. Copy and paste this command into the Google Cloud Shell to create the same keyring.

$ gcloud beta kms keyrings create testkeyring --location global

Create a Key

Now that a KeyRing has been created, it is time to generate a new cryptographic key (called a Key in Cloud KMS) attached to the KeyRing. You must give the Key a name, a rotation period (in days), and when to start the rotation period. The geographic location of the Key (Google Cloud Platform Region) is controlled by the KeyRing.

Click the Create key button. Then, give the Key a keyname of testkey.

Click the Create button.

Optional: You may also create a key using the command-line interface (gcloud) or Cloud KMS API. Copy and paste this command into the Google Cloud Shell to create the same keyring.

$ gcloud beta kms cryptokeys create testkey \
    --location global \
    --keyring testkeyring \
    --purpose encryption

Share Key with the Bookshelf app

Now that a KeyRing and Key have been created, it is time to update the permissions to allow people or resources the ability to interact with Cloud KMS. You may assign an encrypt, decrypt, and/or administer to any Google Account (user or resource). This can be restricted to an individual Key, a set of Keys in a KeyRing, or to all Cloud KMS resources.

In the case of this Bookshelf app codelab, we will provide access to encrypt and decrypt, isolated only to one key using the Google Compute Engine or Google App Engine (GAE)'s built-in Service Account.

In the Google Cloud Console, navigate to the testkey listed in testkeyring within Cloud KMS.

On the righthand side, click on additional settings menu for the testkey. A drop down menu will popup for this individual key, click on the Edit permission button.

A new permissions dialog will pop-up on the righthand side of the screen allowing you to view the active permissions and add a new user (Google Account) or resource (Service Account) with one or more roles.

In the Add members textbox, start typing "appspot" and let the Permissions dialog find the default service account for the GAE app. It will be in the format of <yourproject>@appspot.gserviceaccount.com. Select this Service Account.

Next, select a role for this Service Account. We'll give the app the Encrypter/Decrypter role. Select it from the drop-down menu.

Click the Add button and verify the new Service Account has been added to the list of active permissions for this Key.

Optional: You may also setup permissions using the command-line interface (gcloud) by using the commands below.

SERVICE_ACCOUNT_EMAIL=<PROJECT_ID>@appspot.gserviceaccount.com
gcloud beta kms keyrings add-iam-policy-binding $KEYRING_NAME \
    --location global \
    --member serviceAccount:$SERVICE_ACCOUNT_EMAIL \
    --role roles/cloudkms.cryptoKeyEncrypterDecrypter

Now that you have enabled Cloud Key Management Service (KMS), created a KeyKey, and shared a new Key with the Bookshelf app, it is time to augment the Bookshelf app with encryption.

The diagram below provides an overview of the flow of data between the application, Cloud KMS, and Cloud Storage.

Download additional Bookshelf app files

To expedite the codelab, go to the Google Cloud Shell execute the following commands to download some additional helper files that will be used in the next steps.

$ gsutil cp -r gs://gcp-kms-codelab/Step7/* $HOME/getting-started-java/bookshelf/3-binary-data/

Command output

Copying gs://gcp-kms-codelab/Step7/app.yaml...
Copying gs://gcp-kms-codelab/Step7/pom.xml...                                   
Copying gs://gcp-kms-codelab/Step7/src/main/java/com/example/getstarted/basicactions/CreateBookServlet.java...
/ [3 files][ 10.6 KiB/ 10.6 KiB]    
Copying gs://gcp-kms-codelab/Step7/src/main/java/com/example/getstarted/basicactions/DecryptImageServlet.java...
Copying gs://gcp-kms-codelab/Step7/src/main/java/com/example/getstarted/basicactions/UpdateBookServlet.java...
Copying gs://gcp-kms-codelab/Step7/src/main/java/com/example/getstarted/util/CloudKeyManagementServiceHelper.java...
Copying gs://gcp-kms-codelab/Step7/src/main/java/com/example/getstarted/util/CloudStorageHelper.java...
Copying gs://gcp-kms-codelab/Step7/src/main/webapp/WEB-INF/web.xml...           
Copying gs://gcp-kms-codelab/Step7/src/main/webapp/WEB-INF/web.xml...           
Copying gs://gcp-kms-codelab/Step7/src/main/webapp/list.jsp...                  
Copying gs://gcp-kms-codelab/Step7/src/main/webapp/view.jsp...                  
- [10 files][ 34.7 KiB/ 34.7 KiB]                                               
Operation completed over 10 objects/34.7 KiB. 

This has slightly modified the Bookshelf source code, added some Cloud KMS helper functions, and enabled customer-supplied encryption keys within the write and read calls to GCS.

Now go back to the Google Cloud Shell and run the following command to build and run the Compute Engine application locally. In this case, the encryption keys will be managed by Google Cloud KMS.

Run the Bookshelf app:

$ mvn jetty:run-exploded -DprojectID=MY-PROJECT-ID \
-Dbookshelf.bucket=MY-GCS-BUCKET-NAME \
-Dbookshelf.keyRingName=MY-KMS-KEYRING-NAME \
-Dbookshelf.keyRingLocation=global \
-Dbookshelf.keyName=MY-KMS-KEY-NAME

The Apache Maven client will download dependencies, build the source code, and launch a local version of the Compute Engine app.

Command output

[INFO] Building bookshelf-3 1.0-SNAPSHOT
...
[INFO] Started Jetty Server
...

Once the local server has started, it is time to preview the running application in the web browser. Click on the web preview button within the Google Cloud Shell UI. Select Preview on port 8080, the default.

A new tab will be owned with the running Bookshelf sample application. The Google Compute Engine or Google App Engine app is running locally on the Google Cloud Shell machine, which is, by default, accessible using the web Preview functionality.

Create a new Bookshelf book, define the name, select an image, and save it.

Optional: Create a new Bookshelf book. Understand how data is stored in the new behavior.

  1. Open Google Cloud Storage browser in the Cloud Console and inspect uploaded/stored images.
  2. Does the uploaded image show as "Customer encrypted"?
  3. How is the image being uploaded with a Data Encryption Key (DEK)?
  4. How is the image being decrypted before it is presented to a user?

Now that you have created a few Bookshelf books (one encrypted using Cloud KMS & CSEK, one using default Google encryption at-rest). Let's have a peek at Google Cloud Storage (GCS).

Open the GCS Browser:

Open the Google Cloud Console, navigate to Storage, and view the previously created storage bucket.

Optional:

Congratulations, you have successfully completed this codelab. In the event that you have created some Google Cloud Platform resources, it is time to clean-up after yourself. Please do the following:

Clean-up Google Cloud Shell

In the Google Cloud Shell, there are files associated with this codelab, such as the code associated with the Bookshelf app. You may choose to delete these files by executing the following command in the Google Cloud Shell command-line.

$ rm -r $HOME/getting-started-java

Remove Project or Resources

You may choose on of the two options to clean-up the remaining resources that may be associated with this codelab.

  1. Delete your Google Cloud Platform project entirely (if only used for this codelab and if no longer needed in the future)
  2. Delete individual resources, such as Google Compute Engine or Google App Engine apps, if created accidentally. The codelab did not instruct you to create any resources specifically.