Encrypt Cloud Functions using Customer-managed Encryption Keys (CMEK)

1. Introduction

Overview

Cloud Functions is a lightweight compute solution for developers to create single-purpose, stand-alone functions that respond to Cloud events without needing to manage a server or runtime environment.

You can use Cloud Key Management Service customer-managed encryption keys (CMEK) to protect Cloud Functions and related data at rest. Deploying a function with a CMEK protects the data associated with it by using an encryption key that is in your full control. This type of encryption allows you to meet compliance requirements in certain industries, such as financial services. Because the key is owned by you and is not controlled by Google, no one (including you) can access the data protected by these encryption keys when the keys are disabled or destroyed.

For Cloud Functions, CMEK encrypts the following:

• Function source code uploaded for deployment and stored by Google in Cloud Storage, used in the build process.
• The results of the function build process, including the container image built from your function source code, each instance of the function that is deployed.
• At-rest data for internal event transport channels (1st gen only).

You can find more information on what data is encrypted in the Cloud Function CMEK documentation.

What you'll build

This codelab shows how to deploy a Cloud Function (either 1st gen or 2nd gen) that is encrypted using CMEK. This codelab uses a public Cloud Function, i.e. one that does not require authentication, for demo purposes. You can invoke an authenticated CMEK-enabled function just like any other Cloud Function that requires authentication.

Note: Using CMEK with Cloud Functions 2nd gen is currently in public preview. This codelab uses commands in gcloud beta for Cloud Functions 2nd gen examples.

What you'll learn

• How to create a CMEK key on an existing symmetric key ring
• How to create an Artifact Registry repository
• How to configure CMEK on a Cloud Function for both 1st and 2nd gen

2. Setup and Requirements

Prerequisites

• You are logged into the Cloud Console
• You have previously deployed an HTTP triggered Cloud Function (to verify you have the appropriate roles and APIs enabled)

Activate Cloud Shell

1. From the Cloud Console, click Activate Cloud Shell .

If this is your first time starting Cloud Shell, you're presented with an intermediate screen describing what it is. If you were presented with an intermediate screen, click Continue.

It should only take a few moments to provision and connect to Cloud Shell.

This virtual machine is loaded with all the development tools needed. It offers a persistent 5 GB home directory and runs in Google Cloud, greatly enhancing network performance and authentication. Much, if not all, of your work in this codelab can be done with a browser.

Once connected to Cloud Shell, you should see that you are authenticated and that the project is set to your project ID.

2. Run the following command in Cloud Shell to confirm that you are authenticated:

gcloud auth list

Command output

 Credentialed Accounts
ACTIVE  ACCOUNT
*       <my_account>@<my_domain.com>

To set the active account, run:
    $ gcloud config set account `ACCOUNT`

Note: The gcloud command-line tool is the powerful and unified command-line tool in Google Cloud. It comes preinstalled in Cloud Shell. You will notice its support for tab completion. For more information, see gcloud command-line tool overview.

3. Run the following command in Cloud Shell to confirm that the gcloud command knows about your project:

gcloud config list project

Command output

[core]
project = <PROJECT_ID>

If it is not, you can set it with this command:

gcloud config set project <PROJECT_ID>

Command output

Updated property [core/project].

Note: If you're completing this tutorial outside of Cloud Shell, follow Set up Application Default Credentials.

3. Create a new key ring and key for Cloud Functions

Make sure the Cloud KMS API is enabled by running the following command:

gcloud services enable cloudkms.googleapis.com

First, create environment variables to contain the key ring name, key name, region, and other variables used in this codelab.

KEYRING_NAME="keyring-functions"
REGION="us-central1"
KEY_NAME="key-encrypted-function"
PROJECT_ID=$(gcloud config get-value project)
PROJECT_NUMBER="$(gcloud projects describe $PROJECT_ID --format='value(projectNumber)')"
USER_EMAIL="$(gcloud config list account --format "value(core.account)")"

Next, create a key ring which is the root resource for Cloud KMS keys and key versions.

gcloud kms keyrings create $KEYRING_NAME --location $REGION

Lastly, you can now create a symmetric key in your new key ring within Cloud KMS.

gcloud kms keys create $KEY_NAME --keyring $KEYRING_NAME --location $REGION --purpose "encryption"

4. Create a CMEK-enabled Docker-formatted Artifact Registry repository

In this section, you will create a Docker-formatted repo in Artifact Registry that has CMEK-enabled. This key will be the same key used to deploy your Cloud Function.

First, you'll need the service account for Artifact Registry. You can create it by running this command:

gcloud beta services identity create --service=artifactregistry.googleapis.com --project=$PROJECT_ID

Use the following command to grant the CryptoKey Encrypter/Decrypter IAM role (roles/cloudkms.cryptoKeyEncrypterDecrypter) to the Artifact Registry service account to have permissions to the key:

gcloud kms keys add-iam-policy-binding \
  $KEY_NAME --location $REGION --keyring=$KEYRING_NAME \
  --member serviceAccount:service-$[e-mail] \
  --role roles/cloudkms.cryptoKeyEncrypterDecrypter

And grant the role to the principle that will create the repo in artifact registry, e.g. your current active account. You can verify your current active account by running gcloud auth list.

gcloud kms keys add-iam-policy-binding \
       $KEY_NAME --location $REGION --keyring=$KEYRING_NAME \
       --member user:$USER_EMAIL \
       --role roles/cloudkms.cryptoKeyEncrypterDecrypter

Now you can create a Docker-formatted repo that is CMEK-enabled.

Note: that the region must be the same region as the CMEK key.

REPO_NAME=my-cmek-encrypted-repo 

KEY_FULLPATH=projects/"$PROJECT_ID"/locations/"$REGION"/keyRings/"$KEYRING_NAME"/cryptoKeys/"$KEY_NAME" 

gcloud artifacts repositories create $REPO_NAME \
    --repository-format=docker \
    --location=$REGION \
    --kms-key=$KEY_FULLPATH \
    --async

You can view your new Artifact Registry repo running this command:

gcloud artifacts repositories describe $REPO_NAME --location=$REGION

5. Grant Service Accounts access to the key (2nd gen)

This section covers creating service accounts for 2nd gen functions. If you are creating a 1st gen function, please proceed to the next section.

You must grant several service agents access to the key by granting the CryptoKey Encrypter/Decrypter IAM role (roles/cloudkms.cryptoKeyEncrypterDecrypter). These service agents are used to obtain access to the source code stored in Cloud Storage, store function images in a CMEK-protected repository in Artifact Registry, and to deploy a CMEK-encrypted Cloud Function.

Note if you are using a new project: Some service agents, such as the Cloud Storage service agent, are not initially available when you create a project. Instead, they are activated the first time you access them. And the service agent must be activated prior to assigning permissions to it.

You can activate the Cloud Storage service agent by retrieving its email address:

gcloud storage service-agent –project=$PROJECT_ID

Steps for 2nd gen Functions

1. Grant the Cloud Run service agent access to the key:

CLOUDRUN_SA=service-$[e-mail]

gcloud kms keys add-iam-policy-binding $KEY_NAME \
--keyring=$KEYRING_NAME \
--location=$REGION \
--member=serviceAccount:$CLOUDRUN_SA \
--role=roles/cloudkms.cryptoKeyEncrypterDecrypter

2. Grant the Eventarc service agent access to the key:

EVENTARC_SA=service-$[e-mail]

gcloud kms keys add-iam-policy-binding $KEY_NAME \
--keyring=$KEYRING_NAME \
--location=$REGION \
--member=serviceAccount:$EVENTARC_SA \
--role=roles/cloudkms.cryptoKeyEncrypterDecrypter

3. Grant the Artifact Registry service agent access to the key:

AR_SA=service-$[e-mail]

gcloud kms keys add-iam-policy-binding $KEY_NAME \
--keyring=$KEYRING_NAME \
--location=$REGION \
--member=serviceAccount:$AR_SA \
--role=roles/cloudkms.cryptoKeyEncrypterDecrypter

4. Grant the Cloud Storage service agents access to the key:

STORAGE_SA=service-$[e-mail]

gcloud kms keys add-iam-policy-binding $KEY_NAME \
--keyring=$KEYRING_NAME \
--location=$REGION \
--member=serviceAccount:$STORAGE_SA \
--role=roles/cloudkms.cryptoKeyEncrypterDecrypter

In the next section, you'll see how to create and deploy a CMEK-encrypted function.

6. Grant Service Accounts access to the key (1st gen)

This section covers creating service accounts for 1st gen functions. If you have previously created service accounts for a 2nd gen function, please proceed to the next section.

You must grant several service agents access to the key by granting the CryptoKey Encrypter/Decrypter IAM role (roles/cloudkms.cryptoKeyEncrypterDecrypter). These service agents are used to obtain access to the source code stored in Cloud Storage, store function images in a CMEK-protected repository in Artifact Registry, and to deploy a CMEK-encrypted Cloud Function.

Note if you are using a new project: Some service agents, such as the Cloud Storage service agent, are not initially available when you create a project. Instead, they are activated the first time you access them. And the service agent must be activated prior to assigning permissions to it.

You can activate the Cloud Storage service agent by retrieving its email address:

gcloud storage service-agent –project=$PROJECT_ID

Steps for 1st gen Functions

1. Grant the Cloud Functions service agent access to the key:

FUNCTION_SA=service-$[e-mail]

gcloud kms keys add-iam-policy-binding $KEY_NAME \
--keyring=$KEYRING_NAME \
--location=$REGION \
--member=serviceAccount:$FUNCTION_SA \
--role=roles/cloudkms.cryptoKeyEncrypterDecrypter

2. Grant the Artifact Registry service agent access to the key:

AR_SA=service-$[e-mail]

gcloud kms keys add-iam-policy-binding $KEY_NAME \
--keyring=$KEYRING_NAME \
--location=$REGION \
--member=serviceAccount:$AR_SA \
--role=roles/cloudkms.cryptoKeyEncrypterDecrypter

3. Grant the Cloud Storage service agents access to the key:

STORAGE_SA=service-$[e-mail]

gcloud kms keys add-iam-policy-binding $KEY_NAME \
--keyring=$KEYRING_NAME \
--location=$REGION \
--member=serviceAccount:$STORAGE_SA \
--role=roles/cloudkms.cryptoKeyEncrypterDecrypter

In the next section, you'll see how to create and deploy a CMEK-encrypted function.

7. Create a CMEK-encrypted Function (2nd gen)

This section covers creating 2nd gen functions. You can proceed to the next section for 1st gen instructions.

Now that you have an Artifact Registry repository configured with CMEK enabled and have granted Cloud Functions access to your key, you can now deploy a function that is encrypted using your CMEK key.

Steps for 2nd gen Functions:

Create the source code for the function

Although this codelab uses Node.js, you can use any supported runtime.

First, create a directory and cd into that directory.

mkdir ~/cmek-function-2ndgen && cd $_

Then, create the package.json file.

touch package.json

echo '{
  "dependencies": {
    "@google-cloud/functions-framework": "^2.1.0"
  }
}
' > package.json

Next, create the index.js source file.

touch index.js

echo 'const functions = require("@google-cloud/functions-framework");

functions.http("helloWorld", (req, res) => {
 res.send(`Hello ${req.query.name || req.body.name || "World"}!`);
});' > index.js

Deploy the 2nd gen Cloud Function using CMEK encryption

Note: The example below shows how to deploy a function using sources from your current directory. Make sure you are in the same directory as the source code for your function.

FUNCTION_NAME=protect-me-cmek-2ndgen
ENTRY_POINT=helloWorld

REPO_FULLPATH=projects/"$PROJECT_ID"/locations/"$REGION"/repositories/$REPO_NAME

gcloud beta functions deploy $FUNCTION_NAME  \
--gen2 \
--region $REGION \
--kms-key $KEY_FULLPATH \
--docker-repository $REPO_FULLPATH \
--source . \
--trigger-http \
--allow-unauthenticated \
--runtime nodejs16 \
--entry-point $ENTRY_POINT

You can see the CMEK key from the resulting output by running this command

gcloud functions describe $FUNCTION_NAME –region $REGION | grep kmsKeyName

Test the 2nd gen function

You can test your function by curling it:

FUNCTION_URL="$(gcloud functions describe $FUNCTION_NAME --region $REGION --format='get(serviceConfig.uri)')"

curl $FUNCTION_URL

which results in the following:

Hello World!

As long as the encryption key is enabled, the function will return success to the caller. However, once the encryption key is disabled, the caller will receive an error.

In the next section, you'll see what happens when you invoke the Function after the key has been disabled.

8. Create a CMEK-encrypted Function (1st gen)

This section covers creating 1st gen functions. If you have previously created a 2nd gen function, please proceed to the next section.

Now that you have an Artifact Registry repository configured with CMEK enabled and have granted Cloud Functions access to your key, you can now deploy a function that is encrypted using your CMEK key.

Steps for 1st gen Functions:

Create the source code for the 1st gen function

Although this codelab uses Node.js, you can use any supported runtime.

First, create a directory and cd into that directory.

mkdir ~/cmek-function-1stgen && cd $_

Next, create the package.json file.

touch package.json

echo '{
    "name": "function-cmek-codelab",
    "version": "0.0.1"
}' > package.json

Then, create the index.js source file.

touch index.js

echo "exports.helloWorld = (req, res) => {
    let message = req.query.message || req.body.message || 'Hello World!';
    res.status(200).send(message);
};" > index.js

Deploy the 1st gen Cloud Function using CMEK encryption

Note: The example below shows how to deploy a function using sources from your current directory. Make sure you are in the same directory as the source code for your function.

FUNCTION_NAME=protect-me-cmek-1stgen
ENTRY_POINT=helloWorld

REPO_FULLPATH=projects/"$PROJECT_ID"/locations/"$REGION"/repositories/$REPO_NAME

gcloud functions deploy $FUNCTION_NAME  \
--region $REGION \
--kms-key $KEY_FULLPATH \
--docker-repository $REPO_FULLPATH \
--source . \
--trigger-http \
--allow-unauthenticated \
--runtime nodejs16 \
--entry-point $ENTRY_POINT

You can see the CMEK key from the resulting output by running this command

gcloud functions describe $FUNCTION_NAME –region $REGION | grep kmsKeyName

Test the 1st gen function

You can test your function by curling it:

FUNCTION_URL="$(gcloud functions describe $FUNCTION_NAME --region $REGION --format='get(httpsTrigger.url)')"

curl $FUNCTION_URL

which results in the following:

Hello World!

As long as the encryption key is enabled, the function will return success to the caller. However, once the encryption key is disabled, the caller will receive an error.

In the next section, you'll see what happens when you invoke the Function after the key has been disabled.

9. Invoke a CMEK-encrypted Function where the encryption key has been disabled

In this final section, you will invalidate the key and invoke the Function again to see the resulting error.

Disable the encryption key

You can run this command to disable the key. Since this codelab only creates one version of the key, you'll disable version 1.

gcloud kms keys versions disable 1 \
    --key=$KEY_NAME \
    --keyring=$KEYRING_NAME \
    --location=$REGION

and you should see the resulting info:

algorithm: GOOGLE_SYMMETRIC_ENCRYPTION
createTime: '2023-04-11T03:30:49.111832653Z'
generateTime: '2023-04-11T03:30:49.111832653Z'
name: projects/dogfood-gcf-saraford/locations/us-central1/keyRings/myKeyRing/cryptoKeys/encrypted-function/cryptoKeyVersions/1
protectionLevel: SOFTWARE
state: DISABLED

Note: You must wait a few hours for the change to propagate.

Invoke the Function with a disabled key

Now curl the function again.

curl $FUNCTION_URL

and you will not receive a Hello World response this time.

In the logs for the Cloud Function, you will see

User's CMEK key has been disabled. CMEK key: projects/<PROJECT-NAME>/locations/us-central1/keyRings/myKeyRing/cryptoKeys/encrypted-function

Attempt to view resources when CMEK key is disabled

In this section, you'll see the following resources become unavailable when the CMEK key is disabled:

• Function source code
• Container image build from your source code

For example, visiting the Source tab for the Cloud Function shows an error while fetching the archive. You'll receive a similar error if you attempt to view the .zip file containing the source code directly in Cloud Storage.

In addition, you will not have access to use the container image for the function from Artifact Registry. For example, if you attempt to deploy that container image to Cloud Run, you will receive an error that the image was not found.

Please refer to the CMEK Functions docs for a complete list of encrypted resources.

10. Congratulations

Congratulations, you finished the codelab!

What we've covered

• How to create a CMEK key on an existing symmetric key ring
• How to create an Artifact Registry repository
• How to configure CMEK on a Cloud Function

For more information

You can find more information about Cloud Functions and CMEK in the following links:

• Securing Cloud Functions using CMEK article
• Customer-managed encryption keys documentation