Cloud KMS is a cloud-hosted key management service that lets you manage cryptographic keys for your cloud services the same way you do on-premises. It includes support for encryption, decryption, signing, and verification using a variety of key types and sources including Cloud HSM for hardware-backed keys.

In this tutorial, you will learn how to use the advanced features of Cloud Security and Privacy APIs, including:

You will take abridged data from the Enron Corpus, encrypt it, and load it into Cloud Storage.

You will learn

You will use

Self-paced environment setup

If you don't already have a Google Account (Gmail or Google Apps), you must create one. Sign-in to Google Cloud Platform console (console.cloud.google.com) and create a new project:

Screenshot from 2016-02-10 12:45:26.png

Remember the project ID, a unique name across all Google Cloud projects (the name above has already been taken and will not work for you, sorry!). It will be referred to later in this codelab as PROJECT_ID.

Next, you'll need to enable billing in the Cloud Console in order to use Google Cloud resources.

Running through this codelab shouldn't cost you more than a few dollars, but it could be more if you decide to use more resources or if you leave them running (see "cleanup" section at the end of this document).

New users of Google Cloud Platform are eligible for a $300 free trial.

Start Cloud Shell

In this codelab you will use Cloud Shell, a free virtualized environment running on Google Cloud. From the GCP Console click the Cloud Shell icon on the top right toolbar:

It should only take a few moments to provision and connect to the environment. When it is finished, you should see something like this:

This virtual machine is loaded with all the development tools you'll need. It offers a persistent 5GB home directory, and runs on the Google Cloud, greatly enhancing network performance and authentication. Unless otherwise instructed, run all commands from this shell.

To store data for this tutorial, create a Cloud Storage bucket. Since Cloud Storage bucket names must be globally unique, prefix the bucket name with your project ID:

$ export BUCKET_NAME=${GOOGLE_CLOUD_PROJECT}_enron_emails

Then create the bucket:

$ gsutil mb gs://${BUCKET_NAME}

The Enron Corpus is a large database of over 600,000 emails generated by 158 employees of the Enron Corporation. This data has been copied to the Cloud Storage bucket named gs://enron_emails/.

Download one of the email files locally:

$ gsutil cp gs://enron_emails/allen-p/inbox/1. . # <-- don't forget the dot!

Inspect the downloaded file to verify that it is, in fact, an email:

$ tail 1.

The email will contain a message like:

Attached is the Delta position for 1/18...
# ...

In addition to emails, the bucket also contains images. This tutorial uses both the plaintext emails and images.

Before you can use Cloud KMS, you must first enable the service in your project. This only needs to be done once per project. To enable the Cloud KMS service, run the following command:

$ gcloud services enable cloudkms.googleapis.com \
    --project "${GOOGLE_CLOUD_PROJECT}"

It can take up to a minute to enable. The command will report success when it finishes.

Create a Cloud KMS Key Ring. In Cloud KMS, a Key Ring is a logical collection of cryptographic keys. The Key Ring contains metadata about the keys such as their location. Create a Key Ring named my-keyring in the global region:

$ gcloud kms keyrings create "my-keyring" \
    --location "global"

Now create a Crypto Key named enron-emails with the purpose encryption inside the Key Ring you just created.

$ gcloud kms keys create "enron-emails" \
    --location "global" \
    --keyring "my-keyring" \
    --purpose "encryption"

Open the Cryptographic Keys Web UI and view the newly created resources. This is available under IAM & Admin in the main menu.

You can view and manage Key Rings and Crypto Keys in the Cloud Console.

Encrypt the contents of the email we downloaded earlier using Cloud KMS. This tutorial uses the gcloud command line tool, but you can also encrypt data using the Cloud KMS API.

$ gcloud kms encrypt \
    --location "global" \
    --keyring "my-keyring" \
    --key "enron-emails" \
    --plaintext-file ./1. \
    --ciphertext-file ./1.enc

This will create a new file on disk 1.enc, which contains the encrypted file contents. Now that the data is encrypted with Cloud KMS, upload the encrypted file to the Cloud Storage bucket.

$ gsutil cp ./1.enc gs://${BUCKET_NAME}

Cloud KMS is integrated with Google Cloud Identity and Access Management (IAM). Cloud KMS IAM roles are largely divided into two categories:

For example, the roles/cloudkms.admin role allows the holder to create Key Rings, Crypto Keys, set IAM policies, and perform management-related operations. The roles/cloudkms.cryptoKeyEncrypterDecrypter grants the holder the encrypt and decrypt data, but it does not include management permissions.

IAM roles are inherited by their parent resource. If someone has the roles/cloudkms.admin role on the Google Cloud project, they are an admin of all keys in that project. If they have roles/cloudkms.admin on a Key Ring, they are an admin of all keys in that Key Ring. If they have roles/cloudkms.admin on a single Crypto Key, they are only an admin of that key.

For this exercise, use your identity:

$ export MY_IDENTITY=you@gmail.com # or you@example.com

Assign the identity the ability to manage Cloud KMS resources in the Key Ring created above using the gcloud command line tool:

$ gcloud kms keyrings add-iam-policy-binding "my-keyring" \
    --location "global" \
    --member "user:${MY_IDENTITY}" \
    --role "roles/cloudkms.admin"

The roles/cloudkms.admin role does not include permission to use keys, only to manage them. To grant your identity the ability to encrypt and decrypt data using the enron-emails Crypto Key:

$ gcloud kms keys add-iam-policy-binding "enron-emails" \
    --location "global" \
    --keyring "my-keyring" \
    --member "user:${MY_IDENTITY}" \
    --role "roles/cloudkms.cryptoKeyEncrypterDecrypter"

Note that the first command grants the roles/cloudkms.admin role on the Key Ring, which would include any child Crypto Key resources. The second command grants roles/cloudkms.cryptoKeyEncrypterDecrypter, but only on the enron-emails Crypto Key.

You can also view and assign roles in the Cloud KMS UI in the Cloud Console.

In addition to encrypting a single file, you can also encrypt a collection of files with some scripting. This example streams all emails for allen-p, encrypts them, and uploads the resulting encrypted values to the Cloud Storage bucket:

DIR="gs://enron_emails/allen-p"
for file in $(gsutil ls ${DIR}/**); do
  ENC_NAME="$(basename ${file}).enc"
  gsutil cat ${file} \
  | \

  gcloud kms encrypt \
    --location "global" \
    --keyring "my-keyring" \
    --key "enron-emails" \
    --plaintext-file - \
    --ciphertext-file - \
    | \

    gsutil -q cp - gs://${BUCKET_NAME}/${ENC_NAME}
    echo "Copied ${ENC_NAME}"
done

This iterates over all the files in the given directory in the sample Cloud Storage bucket, encrypts them using Cloud KMS, and uploads them to your Google Cloud Storage.

After the script completes, you can view the encrypted files in the Cloud Storage UI. You should see something like this:

Cloud Audit Logging has two log streams – Admin Activity Logs and Data Access Logs – which are generated by Google Cloud services to help you answer the question of "who did what, where, and when?" within your Google Cloud Platform projects.

To view the activity for any resource in KMS, click the key ring and select View Activity. This will take you to the Cloud Activity UI, where you should see the creation and all modifications made to the KeyRing.

You have successfully encrypted data using Cloud KMS and stored encrypted data in Cloud Storage.

Cleanup

Release the resources created during this tutorial. Delete the Cloud Storage bucket created earlier:

$ gsutil rm -r gs://${BUCKET_NAME}

Cloud KMS resources cannot be deleted. However, you can destroy the key material so it cannot be used again:

$ gcloud kms keys versions destroy "1" \
    --location "global" \
    --key "enron-emails" \
    --keyring "my-keyring"

We covered

Next steps

Learn more

License

This work is licensed under a Creative Commons Attribution 2.0 Generic License.