In this codelab, you will focus on using Secret Manager in Python.
Secret Manager allows you to store, manage, and access secrets as binary blobs or text strings. With the appropriate permissions, you can view the contents of the secret.
Secret Manager works well for storing configuration information such as database passwords, API keys, or TLS certificates needed by an application at runtime.
What you'll learn
- How to use Cloud Shell
- How to install the Secret Manager client library for Python
- How to create and access secrets using the Python client library
- How to access secrets in Cloud Functions using the Python client library
What you'll need
Survey
How will you use this tutorial?
How would you rate your experience with Python?
How would you rate your experience with using Google Cloud services?
Self-paced environment setup
- Sign in to Cloud Console and create a new project or reuse an existing one. (If you don't already have a Gmail or G Suite account, you must create one.)
Remember the project ID, a unique name across all Google Cloud projects (the name above has already been taken and will not work for you, sorry!). It will be referred to later in this codelab as PROJECT_ID
.
- Next, you'll need to enable billing in Cloud Console in order to use Google Cloud resources.
Running through this codelab shouldn't cost much, if anything at all. Be sure to to follow any instructions in the "Cleaning up" section which advises you how to shut down resources so you don't incur billing beyond this tutorial. New users of Google Cloud are eligible for the $300USD Free Trial program.
Start Cloud Shell
While Google Cloud can be operated remotely from your laptop, in this codelab you will be using Google Cloud Shell, a command line environment running in the Cloud.
Activate Cloud Shell
- From the Cloud Console, click Activate Cloud Shell
.
If you've never started Cloud Shell before, you'll be presented with an intermediate screen (below the fold) describing what it is. If that's the case, click Continue (and you won't ever see it again). Here's what that one-time screen looks like:
It should only take a few moments to provision and connect to Cloud Shell.
This virtual machine is loaded with all the development tools you'll need. It offers a persistent 5GB home directory and runs in Google Cloud, greatly enhancing network performance and authentication. Much, if not all, of your work in this codelab can be done with simply a browser or your Chromebook.
Once connected to Cloud Shell, you should see that you are already authenticated and that the project is already set to your project ID.
- Run the following command in Cloud Shell to confirm that you are authenticated:
gcloud auth list
Command output
Credentialed Accounts ACTIVE ACCOUNT * <my_account>@<my_domain.com> To set the active account, run: $ gcloud config set account `ACCOUNT`
gcloud config list project
Command output
[core] project = <PROJECT_ID>
If it is not, you can set it with this command:
gcloud config set project <PROJECT_ID>
Command output
Updated property [core/project].
Before you can begin using the Secret Manager API, you must enable the API. Using Cloud Shell, you can enable the API with the following command:
gcloud services enable secretmanager.googleapis.com
You should see output like this:
Operation "operations/acf.cc11852d-40af-47ad-9d59-477a12847c9e" finished successfully.
Install the Secret Manager Client Library:
pip3 install --user google-cloud-secret-manager==2.0.0
For part of this tutorial, you'll use an interactive Python interpreter called IPython, which is preinstalled in Cloud Shell. Start a session by running ipython
in Cloud Shell:
ipython
You should see something like this:
Python 3.7.3 (default, Jul 25 2020, 13:03:44) Type 'copyright', 'credits' or 'license' for more information IPython 7.18.1 -- An enhanced Interactive Python. Type '?' for help. In [1]:
A secret contains one or more secret versions. They can be created using the gcloud
command-line, but they can also be created using Python.
In order to use a secret, you first need to create the secret with the name of the secret, then you add a version of the secret, being the value of the secret.
Set your Project ID within IPython:
PROJECT_ID = "<PROJECT_ID>"
Create a secret
Copy the following code into your IPython session:
from google.cloud import secretmanager
def create_secret(secret_id):
# Create the Secret Manager client.
client = secretmanager.SecretManagerServiceClient()
# Build the resource name of the parent project.
parent = f"projects/{PROJECT_ID}"
# Build a dict of settings for the secret
secret = {'replication': {'automatic': {}}}
# Create the secret
response = client.create_secret(secret_id=secret_id, parent=parent, secret=secret)
# Print the new secret name.
print(f'Created secret: {response.name}')
Call the function to create a new secret called my_secret_value
:
create_secret("my_secret_value")
You should see the following output:
Created secret: projects/<PROJECT_NUM>/secrets/my_secret_value
Add a secret version
Now that the secret exists, you can assign it a value by creating a version.
Copy the following code into your IPython session:
def add_secret_version(secret_id, payload):
# Create the Secret Manager client.
client = secretmanager.SecretManagerServiceClient()
# Build the resource name of the parent secret.
parent = f"projects/{PROJECT_ID}/secrets/{secret_id}"
# Convert the string payload into a bytes. This step can be omitted if you
# pass in bytes instead of a str for the payload argument.
payload = payload.encode('UTF-8')
# Add the secret version.
response = client.add_secret_version(parent=parent, payload={'data': payload})
# Print the new secret version name.
print(f'Added secret version: {response.name}')
Call the function to create a add a new secret version:
add_secret_version("my_secret_value", "Hello Secret Manager")
You should see the following output:
Added secret version: projects/<PROJECT_NUM>/secrets/my_secret_value/versions/1
Secrets can have multiple versions. Call the function again with a different value:
add_secret_version("my_secret_value", "Hello Again, Secret Manager")
You should see the following output:
Added secret version: projects/<PROJECT_NUM>/secrets/my_secret_value/versions/2
Notice how the new version of our secret is significantly longer than our original. This attribute will be referenced later.
Accessing a secret version returns the secret contents, as well as additional metadata about the secret version. When you access a secret version, you can either specify a specific version, or just ask for the latest version by specifying "latest".
Secrets should be kept secret. Store database credentials as secrets then use them to authenticate, or store certifications and use them; but do not directly print out your secrets, as this defeats the purpose of keeping them secret.
You're going to perform operations on our secrets, assessing its value without printing it out directly. Instead you'll print out a hash of the value of the secret.
Copy the following code into your IPython session:
def access_secret_version(secret_id, version_id="latest"):
# Create the Secret Manager client.
client = secretmanager.SecretManagerServiceClient()
# Build the resource name of the secret version.
name = f"projects/{PROJECT_ID}/secrets/{secret_id}/versions/{version_id}"
# Access the secret version.
response = client.access_secret_version(name=name)
# Return the decoded payload.
return response.payload.data.decode('UTF-8')
import hashlib
def secret_hash(secret_value):
# return the sha224 hash of the secret value
return hashlib.sha224(bytes(secret_value, "utf-8")).hexdigest()
Call the function to retrieve the secret as a hash of it's value:
secret_hash(access_secret_version("my_secret_value"))
You should see output that resembles a hash (the exact value may not match this output):
83f8a4edb555cde4271029354395c9f4b7d79706ffa90c746e021d11
Since you did not specify a version, the latest value was retrieved.
Call the function adding the expected version number to confirm:
secret_hash(access_secret_version("my_secret_value", version_id=2))
You should see the same output as the last command:
83f8a4edb555cde4271029354395c9f4b7d79706ffa90c746e021d11
Call the function again, but this time specifying the first version:
secret_hash(access_secret_version("my_secret_value", version_id=1))
You should see a different hash this time, indicating a different output:
9a3fc8b809ddc611c82aee950c636c7557e220893560ec2c1eeeb177
You can make use of secrets within many parts of Google Cloud. In this section, you'll focus on Cloud Functions, Google's event-driven serverless compute offering.
If you are interested in using Python in Cloud Functions, you can follow the HTTP Google Cloud Functions in Python Codelab.
Close IPython by calling the exit
function:
exit
You should be returned to your Cloud Shell:
yourname@cloudshell:~ (<PROJECT_ID>)$
Before you can begin using the Cloud Functions API, you must enable the API. Using Cloud Shell, you can enable the API with the following command:
gcloud services enable cloudfunctions.googleapis.com
Create a new folder to build our function, creating empty files to write to:
mkdir secret-manager-api-demo cd secret-manager-api-demo touch main.py touch requirements.txt
Open the code editor from the top right side of the Cloud Shell:
Navigate to the main.py
file inside the secret-manager-api-demo
folder. This is where you'll be putting all your code.
While storing and retrieving secret values from the command line or IPython terminal is useful, it's much more useful to be able to access these secrets within a function.
Using the access_secret_version
function you created earlier, you can use that as a base for your Cloud Function.
Copy the following code into the main.py
file:
main.py
import os
from google.cloud import secretmanager
project_id = os.environ["GCP_PROJECT"]
client = secretmanager.SecretManagerServiceClient()
name = f"projects/{project_id}/secrets/my_secret_value/versions/latest"
response = client.access_secret_version(name=name)
my_secret_value = response.payload.data.decode("UTF-8")
def secret_hello(request):
if "Again" in my_secret_value:
return "We meet again!\n"
return "Hello there.\n"
Before you can deploy your function, you need to finalize the setup of the environment. This requires that you set up your function dependency.
Create a new file called requirements.txt
, and add the google-cloud-secret-manager
package to it:
requirements.txt
google-cloud-secret-manager==2.0.0
You should now have a folder containing just a main.py
and a requirements.txt
.
Allowing access to your secret
Before you can deploy your function, you need to allow Cloud Functions the ability to access your secret.
Switch back to the terminal:
Grant access to the Cloud Functions Service Account to access your secret:
export PROJECT_ID=$(gcloud config get-value core/project) gcloud secrets add-iam-policy-binding my_secret_value \ --role roles/secretmanager.secretAccessor \ --member serviceAccount:${PROJECT_ID}@appspot.gserviceaccount.com
You should see the following output:
Updated IAM policy for secret [my_secret_value]. bindings: - members: - serviceAccount:<PROJECT_ID>@appspot.gserviceaccount.com role: roles/secretmanager.secretAccessor etag: BwWiRUt2oB4= version: 1
Given your setup in the previous sections, you can now deploy and test your Cloud Function.
Within the folder containing just the two files you created, deploy the function:
gcloud functions deploy secret_hello \ --runtime python37 \ --trigger-http \ --allow-unauthenticated
You should see the following output (truncated):
Deploying function (may take a while - up to 2 minutes)...done. ... entryPoint: secret_hello httpsTrigger: url: https://<REGION>-<PROJECT_ID>.cloudfunctions.net/secret_hello ... status: ACTIVE ...
Retrieve the URL of your function (httpsTrigger.url
metadata) with the following command:
FUNCTION_URL=$(gcloud functions describe secret_hello --format 'value(httpsTrigger.url)')
Now, test the function can be accessed with the expected return value, by calling your function:
curl $FUNCTION_URL
You should see the following output:
We meet again!
You learned how to use the Secret Manager API using Python!
Clean up
To avoid incurring charges to your Google Cloud account for the resources used in this tutorial:
- In the Cloud Console, go to the Manage resources page.
- In the project list, select your project then click Delete.
- In the dialog, type the project ID and then click Shut down to delete the project.
Learn more
- Secret Manager: https://cloud.google.com/secret-manager/
- Python on Google Cloud: https://cloud.google.com/python/
- Cloud Client Libraries for Python: https://googlecloudplatform.github.io/google-cloud-python/
License
This work is licensed under a Creative Commons Attribution 2.0 Generic License.