Cloud Foundation Toolkit 101

1. Introduction to CFT 101

Last Updated: 2020-03-03

What is the Cloud Foundation Toolkit?

In essence, CFT provides best-practice templates to quickly get started in Google Cloud Platform. In this tutorial, you will learn how to contribute to the Cloud Foundation Toolkit.

What you'll need

• A GitHub account.
• Docker installed on your machine ( Mac install, Windows install)
• Code Editor for editing code (Example: Visual Studio Code)
• Basic familiarity with Git and GitHub
• Some experience with Terraform and infrastructure as code
• Permission to grant the Project Creator role to a Service Account

What you'll build

In this codelab, you're going to learn how to contribute to the Cloud Foundation Toolkit (CFT).

You will:

• Setup a dev environment for contributing to CFT
• Add a feature to a CFT module
• Add tests for the added feature
• Execute integration tests in CFT
• Execute lint tests
• Commit code to GitHub and submit a Pull Request (PR)

You'll execute all the above steps by adding a new feature to the Google Cloud Storage CFT module. You will be adding a label called the "silly_label" which will be automatically added to all the buckets created through the GCS CFT module. You will also get to write tests to validate your feature and ensure end to end integration.

2. Setup Dev Environment

If you want, you can utilise the Cloud Shell for your development purposes. If you don't want to use Cloud Shell for contributing to CFT, you can setup your dev environment on your machine.

Set up Git

GitHub is based on an open source version control system (VCS) called Git. Git is responsible for everything GitHub-related that happens locally on your machine or your Cloud Shell.

1. When you use the Cloud Shell, you don't need to install git as it comes pre-installed.

$ git --version
# This will display the git version on the Cloud Shell.

If you are setting up your dev environment on your machine you need to install Git.

Download and Install the latest version of Git

Setting your username and email in Git

Git uses a username to associate commits with an identity. The Git username is not the same as your GitHub username.

You can change the name that is associated with your Git commits using the git config command. Changing the name associated with your Git commits using git config will only affect future commits and will not change the name used for past commits.

Setting your Git username for every repository on your computer

For Mac/Linux: Open a Terminal

For Windows: Open Git Bash

Set a Git username:

$ git config --global user.name "Your Name"

$ git config --global user.email "[e-mail]"

Confirm that you have set the Git username correctly:

$ git config --list

> Your Name

> Your Email

You have set up Git successfully and you should be able to fork, create and clone branches. We will be using Git extensively in this Codelab.

3. Fork CFT's GCS Repository

Fork a CFT repository

You set up Git on your local machine or your Cloud Shell in the earlier step. Now you need to fork the Google Cloud Storage CFT repo to start contributing.

A fork is a copy of a repository. Forking a repository allows you to freely experiment with changes without affecting the original project.

Most commonly, forks are used to either propose changes to someone else's project or to use someone else's project as a starting point for your own idea.

For example, you can use forks to propose changes related to fixing a bug. To fix a bug, you can:

• Fork the repository.
• Make the fix.
• Submit a pull request to the project owner.

Steps for forking a CFT repo:

1. Open your web browser and navigate to terraform-google-modules/terraform-google-cloud-storage repository. We will be using this repo for the entire Codelab.
2. In the top-right corner of the page, click Fork.

3. You will be presented with an option of where you want to have the fork, choose your profile and the repo will be forked.

Clone your fork locally

The fork you created is a copy of the GCS module repository. You will now clone this repository to your local environment to add your new feature.

Steps to clone your fork:

1. Open your web browser and navigate to your fork on terraform-google-modules/terraform-google-cloud-storage.
2. In the top-right corner you will find the "Clone or download" button, click on it.

3. After you have clicked on the "Clone or download" button, click on the "Notepad" icon to copy the URL of the fork. You will use this URL to clone your fork to your local environment.

4. Go to a terminal in your VSCode or your machine and clone the fork.

$ git clone <url>
# This command will clone your fork locally.
# Paste the copied URL from the previous step.

5. Now that you have cloned your fork locally, you should go into your repo, create a new branch off the fork and make code changes to the temporary branch.

By convention, you can name your branch as follows:

• For feature requests: feature/feature-name
• For internal updates, internal/change-name
• For bug fixes: bugfix/issue-name

Since you are adding a new feature, you can call your temporary branch feature/silly_label

$ cd terraform-google-cloud-storage
# This command takes you into the cloned directory on your local machine.

$ git branch
# This command tells your current branch
# When you run this for the first time after you have cloned, your 
# output should say "master", that is your fork.

$ git checkout -b feature/silly_label
# This command creates a new branch on your fork and switches your 
# branch to the newly created branch.

$ git branch
# This command will confirm your current branch to be "feature/silly_label"

You are now all setup for starting work on the Cloud Foundation Toolkit!

4. Create a test environment

The standard CFT development process is based around using an isolated test project for testing. This step will guide you through creating the test project (based on a standard configuration) via a service account.

0. Install Docker Engine

If you are using your machine for development purposes, you need to install Docker Engine.

Note: If you are using Cloud Shell, you don't need to install Docker as Cloud Shell already has it.

1. Install Google Cloud SDK

You don't need to install Google Cloud SDK if you are using GCP Cloud Shell.

Go to Google Cloud SDK and download the interactive installer for your platform.

2. Set configuration

In order to create a test environment, you will need a Google Cloud Organization, a test folder, and a billing account. These values need to be set via environment variables:

export TF_VAR_org_id="your_org_id"
export TF_VAR_folder_id="your_folder_id"
export TF_VAR_billing_account="your_billing_account_id"

3 Set up your Service Account

Before creating a test environment, you need to download a service account key to your test environment. This service account will need Project Creator, Billing Account User, and Organization Viewer roles. These steps help you create a new service account, but you could also reuse an existing account.

3.1 Create or Select Seed GCP Project

Before creating your service account, you need to select a project to host it in. You can also create a new project.

gcloud config set core/project YOUR_PROJECT_ID

3.2 Enable Google Cloud APIs

Enable the following Google Cloud APIs on your seed project:

gcloud services enable cloudresourcemanager.googleapis.com
gcloud services enable iam.googleapis.com
gcloud services enable cloudbilling.googleapis.com

3.3 Create Service Account

Create a new service account to manage the test environment:

# Creating a service account for CFT.
gcloud iam service-accounts create cft-onboarding \
  --description="CFT Onboarding Terraform Service Account" \
  --display-name="CFT Onboarding"

# Assign SERVICE_ACCOUNT environment variable for later steps
export SERVICE_ACCOUNT=cft-onboarding@$(gcloud config get-value core/project).iam.gserviceaccount.com

Verify your service account is created:

gcloud iam service-accounts list --filter="EMAIL=${SERVICE_ACCOUNT}"

3.4 Grant Project Creator, Billing Account User, and Organization Viewer roles to the Service Account:

gcloud resource-manager folders add-iam-policy-binding ${TF_VAR_folder_id} \
  --member="serviceAccount:${SERVICE_ACCOUNT}" \
  --role="roles/resourcemanager.projectCreator"
gcloud organizations add-iam-policy-binding ${TF_VAR_org_id} \
  --member="serviceAccount:${SERVICE_ACCOUNT}" \
  --role="roles/billing.user"
gcloud organizations add-iam-policy-binding ${TF_VAR_org_id} \
  --member="serviceAccount:${SERVICE_ACCOUNT}" \
  --role="roles/resourcemanager.organizationViewer"

Now you have a Service Account which can be used for managing the test environment.

4. Assign Billing Account User role on the billing account resource

4.1 Fetch the billing account IAM policy

Download the existing IAM policy bindings on the billing account

gcloud beta billing accounts get-iam-policy ${TF_VAR_billing_account} | tee policy.yml

4.2 Update the policy to include the Service Account

Update the policy.yml file to add a new binding for the service account with the roles/billing.user role

bindings:
- members:
  - serviceAccount:cft-onboarding@<YOUR_PROJECT_ID>.iam.gserviceaccount.com
  role: roles/billing.user

4.3 Update the billing account policy

Apply the changes upon the billing account

gcloud beta billing accounts set-iam-policy ${TF_VAR_billing_account} policy.yml

5. Prepare Terraform Credential

In order to create the test environment, you need to download the service account key into your shell.

5.1 Service Account Key

Create and download a service account key for Terraform

gcloud iam service-accounts keys create cft.json --iam-account=${SERVICE_ACCOUNT}

5.2 Setup Terraform Credential

Supply the key to Terraform using the environment variable SERVICE_ACCOUNT_JSON, setting the value to the contents of your service account key.

export SERVICE_ACCOUNT_JSON=$(< cft.json)

6. Create test project for Terraform deployments

Now that everything is prepared, you can create the test project with a single command. Run this command from the terraform-google-cloud-storage directory root:

make docker_test_prepare

You will see the below output when you run make docker_test_prepare , at the end you will receive the test project_id that has been created where you will deploy and test your Cloud Storage module with your new feature.

macbookpro3:terraform-google-cloud-storage user$ make docker_test_prepare
docker run --rm -it \
                -e SERVICE_ACCOUNT_JSON \
                -e TF_VAR_org_id \
                -e TF_VAR_folder_id \
                -e TF_VAR_billing_account \
                -v /Users/cft/terraform-google-cloud-storage:/workspace \
                gcr.io/cloud-foundation-cicd/cft/developer-tools:0.8.0 \
                /usr/local/bin/execute_with_credentials.sh prepare_environment
Activated service account credentials for: [cft-onboarding@<project_id>.iam.gserviceaccount.com]
Activated service account credentials for: [cft-onboarding@<project_id>.iam.gserviceaccount.com]
Initializing modules...

Initializing the backend...

Initializing provider plugins...

The following providers do not have any version constraints in configuration,
so the latest version was installed.

To prevent automatic upgrades to new major versions that may contain breaking
changes, it is recommended to add version = "..." constraints to the
corresponding provider blocks in configuration, with the constraint strings
suggested below.

* provider.google-beta: version = "~> 3.9"
* provider.null: version = "~> 2.1"
* provider.random: version = "~> 2.2"

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
module.project.module.project-factory.null_resource.preconditions: Refreshing state... [id=8723188031607443970]
module.project.module.project-factory.null_resource.shared_vpc_subnet_invalid_name[0]: Refreshing state... [id=5109975723938185892]
module.project.module.gsuite_group.data.google_organization.org[0]: Refreshing state...
module.project.module.project-factory.random_id.random_project_id_suffix: Refreshing state... [id=rnk]
module.project.module.project-factory.google_project.main: Refreshing state... [id=<project-id>]
module.project.module.project-factory.google_project_service.project_services[0]: Refreshing state... [id=<project-id>/storage-api.googleapis.com]
module.project.module.project-factory.google_project_service.project_services[1]: Refreshing state... [id=<project-id>/cloudresourcemanager.googleapis.com]
module.project.module.project-factory.google_project_service.project_services[2]: Refreshing state... [id=<project-id>/compute.googleapis.com]
module.project.module.project-factory.data.null_data_source.default_service_account: Refreshing state...
module.project.module.project-factory.google_service_account.default_service_account: Refreshing state... [id=projects/ci-cloud-storage-ae79/serviceAccounts/project-service-account@<project-id>.iam.gserv
iceaccount.com]
module.project.module.project-factory.google_project_service.project_services[3]: Refreshing state... [id=<project-id>/serviceusage.googleapis.com]
module.project.module.project-factory.null_resource.delete_default_compute_service_account[0]: Refreshing state... [id=3576396874950891283]
google_service_account.int_test: Refreshing state... [id=projects/<project-id>/serviceAccounts/cft-onboarding@<project-id>.iam.gserviceaccount.com]
google_service_account_key.int_test: Refreshing state... [id=projects/<project-id>/serviceAccounts/cft-onboarding@<project-id>.iam.gserviceaccount.com/keys/351009a1e011e88049ab2097994d1c627a61
6961]
google_project_iam_member.int_test[1]: Refreshing state... [id=<project-id>/roles/iam.serviceAccountUser/serviceaccount:cft-onboarding@<project-id>.iam.gserviceaccount.com]
google_project_iam_member.int_test[0]: Refreshing state... [id=<project-id>/roles/storage.admin/serviceaccount:cft-onboarding@<project-id>.iam.gserviceaccount.com]

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

project_id = <test-project-id>
sa_key = <sensitive>
Found test/setup/make_source.sh. Using it for additional explicit environment configuration.

You have now created a test project which is referenced by project_id as you can see on the console output. Your development and test environment is setup.

5. Add a new feature to CFT module

Now your development and test environment are setup, let's start adding your "silly_label" feature to the google-cloud-storage CFT module.

Make sure you are in the terraform-google-cloud-storage and open the main.tf file as you see below in the folder structure.

Since "silly_label" is a label, you will add the feature in line 27 in the variable "labels" in main.tf, as you see below:

terraform-google-cloud-storage/main.tf

resource "google_storage_bucket" "buckets" {
 <...>
 storage_class = var.storage_class
 // CODELAB:Add silly label in labels variable
 labels        = merge(var.labels, { name = replace("${local.prefix}${lower(element(var.names, count.index))}", ".", "-") }, { "silly" = var.silly_label })
 force_destroy = lookup(
 <...>
}

Now, you will add the silly_label variable in the variables.tf that you see in the above folder structure.

Copy paste the below code and add it to line 29 in variables.tf and ensure you have a new line character above and below the variable block you add.

terraform-google-cloud-storage/variables.tf

variable "names" {
 description = "Bucket name suffixes."
 type        = list(string)
}

// CODELAB: Add "silly_label" variable to variables.tf between "names" and "location"
variable "silly_label" {
 description = "Sample label for bucket."
 type        = string
}

variable "location" {
 description = "Bucket location."
 default     = "EU"
}

6. Add a new feature to example of storage bucket

You have added your feature to the main.tf of the module and now you will test your added feature through an example.

The "silly_label" will need to be added to the examples/multiple-buckets/main.tf

This example will be used by a fixture in the next step to perform integration tests.

Copy paste the below variable silly_label line to line 27 in main.tf at terraform-google-cloud-storage/examples/multiple-buckets/ as seen in the folder structure:

terraform-google-cloud-storage/examples/multiple-buckets/main.tf

module "cloud_storage" {
 <...>
 // CODELAB: Add "silly_label" as an example to main.tf.
 silly_label        = "awesome"

 <..>
}

7. Write an Inspec test to check feature

You have added your feature to the main.tf of the module and then added the feature to multiple_buckets example to be tested through the fixture. You need to test your feature via an InSpec integration test written in Ruby.

You will add your new tests in gsutil.rb file found in the below folder structure:

You added the "silly_label" on all the buckets being created through the multiple_buckets module and now you need to write tests to test out the new feature.

In the below code, you are getting the label of each bucket through the gsutil command and then you check the returned output from the command.

terraform-google-cloud-storage/test/integration/multiple-buckets/controls/gsutil.rb

control "gsutil" do
 <..>

# CODELAB: Copy paste the below test in gsutil.rb to test silly_label feature.

# command to get the labels for bucket_1
describe command("gsutil label get gs://#{attribute("names_list")[0]}") do
   //check if the command gave a valid response
   its(:exit_status) { should eq 0 }
   its(:stderr) { should eq "" }

   //parse the command's output into JSON
   let!(:data) do
     if subject.exit_status == 0
         JSON.parse(subject.stdout)
     else
         {}
     end
   end

   # check if bucket_1 has the new "silly" label with the value "awesome"
   describe "bucket_1" do
   it "has label" do
       data.each do |bucket|
           expect(data["silly"]).to include("awesome")
       end
     end
   end
 end

8. Execute integration tests in CFT

Integration Testing

Integration tests are used to verify the behaviour of the root module, submodules, and example modules. Additions, changes, and fixes should be accompanied with tests.

The integration tests are run using Kitchen, Kitchen-Terraform, and InSpec. These tools are packaged within a Docker image for convenience.

The general strategy for these tests is to verify the behaviour of the example modules, thus ensuring that the root module, submodules, and example modules are all functionally correct.

In interactive execution, you execute each step via multiple commands.

1. Run make docker_run to start the testing Docker container in interactive mode.

Make is a build automation tool that automatically builds executable programs and libraries from source code by reading files called Makefiles which specify how to derive the target program. When you make file changes the docker container must be updated automatically.

When you run make docker_run, you create a workspace in your docker container and activate the credentials for your service account. The workspace will be used in the next steps to run tests.

You will see the below output in your terminal:

Activated service account credentials for: [cft@<PROJECT_ID>.iam.gserviceaccount.com]

2. Run kitchen_do list to list all the instances in your workspace which contain integration tests.

    You will see the below output in your terminal.
   

[root@<CONTAINER_ID> workspace]# kitchen_do list
Automatically setting inputs from outputs of test/setup
Found test/source.sh. Using it for additional explicit environment configuration.
Activated service account credentials for: [cft@<PROJECT_ID>.iam.gserviceaccount.com]
Instance                  Driver     Provisioner  Verifier   Transport  Last Action  Last Error
multiple-buckets-default  Terraform  Terraform    Terraform  Ssh        Verified     <None>

3. Run kitchen_do create <EXAMPLE_NAME> to initialize the working directory for an example module.

This step initializes kitchen and initializes terraform in the workspace.

You will see the below output in your terminal.

[root@<CONTAINER_ID> workspace]# kitchen_do create multiple-buckets-default
Automatically setting inputs from outputs of test/setup
Found test/source.sh. Using it for additional explicit environment configuration.
Activated service account credentials for: [cft@<PROJECT_ID>.iam.gserviceaccount.com]
-----> Starting Kitchen (v1.24.0)
-----> Creating <multiple-buckets-default>...
       Terraform v0.12.12
       + provider.google v3.10.0
       
       Your version of Terraform is out of date! The latest version
       is 0.12.21. You can update by downloading from www.terraform.io/downloads.html
$$$$$$ Running command `terraform init -input=false -lock=true -lock-timeout=0s  -upgrade -force-copy -backend=true  -get=true -get-plugins=true -verify-plugins=true` in directory /workspace/test/fi
xtures/multiple_buckets
       Upgrading modules...
       - example in ../../../examples/multiple_buckets
       - example.cloud_storage in ../../..
       
       Initializing the backend...
       
       Initializing provider plugins...
       - Checking for available provider plugins...
       - Downloading plugin for provider "google" (hashicorp/google) 2.18.1...
       - Downloading plugin for provider "random" (hashicorp/random) 2.2.1...
       
       Terraform has been successfully initialized!
$$$$$$ Running command `terraform workspace select kitchen-terraform-multiple-buckets-default` in directory /workspace/test/fixtures/multiple_buckets
       Finished creating <multiple-buckets-default> (0m11.01s).
-----> Kitchen is finished. (0m12.62s)

4. Run kitchen_do converge <EXAMPLE_NAME> to apply the example module.

This step applies the terraform workspace that was created in the previous step to the GCP project created earlier in the Codelab.

You will see the below output in your terminal.

[root@<CONTAINER_ID> workspace]# kitchen_do converge multiple-buckets-default
Automatically setting inputs from outputs of test/setup
Found test/source.sh. Using it for additional explicit environment configuration.
Activated service account credentials for: [cft@<YOUR_PROJECT_ID>.iam.gserviceaccount.com]
-----> Starting Kitchen (v1.24.0)
-----> Converging <multiple-buckets-default>...
       Terraform v0.12.20
       + provider.google v3.9.0
       
       Your version of Terraform is out of date! The latest version
       is 0.12.21. You can update by downloading from https://www.terraform.io/downloads.html
$$$$$$ Running command `terraform workspace select kitchen-terraform-multiple-buckets-default` in directory /workspace/test/fixtures/multiple_buckets
$$$$$$ Running command `terraform get -update` in directory /workspace/test/fixtures/multiple_buckets
       - example in ../../../examples/multiple_buckets
       - example.cloud_storage in ../../..
$$$$$$ Running command `terraform validate   ` in directory /workspace/test/fixtures/multiple_buckets
       Success! The configuration is valid.
       
$$$$$$ Running command `terraform apply -lock=true -lock-timeout=0s -input=false -auto-approve=true  -parallelism=10 -refresh=true  ` in directory /workspace/test/fixtures/multiple_buckets
       random_pet.main: Creating...
       random_pet.main: Creation complete after 0s [id=<BUCKET-ID>]
       module.example.module.cloud_storage.google_storage_bucket.buckets[0]: Creating...
       module.example.module.cloud_storage.google_storage_bucket.buckets[1]: Creating...
       module.example.module.cloud_storage.google_storage_bucket.buckets[1]: Creation complete after 3s [id=<BUCKET-ID-01>]
       module.example.module.cloud_storage.google_storage_bucket.buckets[0]: Creation complete after 3s [id=<BUCKET-ID-02>]
       
       Apply complete! Resources: 3 added, 0 changed, 0 destroyed.
       
       Outputs:
       
       names = {
         "one" = "<BUCKET-ID-01>"
         "two" = "<BUCKET-ID-02>"
       }
       names_list = [
         "<BUCKET-NAME-01>",
         "<BUCKET-NAME-02>",
       ]
       project_id = ci-cloud-storage-ae79
       Finished converging <multiple-buckets-default> (0m7.17s).
-----> Kitchen is finished. (0m8.77s)

5. Run kitchen_do verify <EXAMPLE_NAME> to test the example module.

This step will run through the gsutils.rb file which contains tests for the multiple_buckets module. Each test has a gsutil command which will be run against the test project that you created earlier using the service account credentials setup.

If you get any errors, you will see what was expected and what was received by the command for the test.

You will see the below output in your terminal.

multiple_buckets local: Verifying

Profile: multiple_buckets
Version: (not specified)
Target:  local://

  ✔  gsutil: gsutil
     ✔  Command: `gsutil ls -p <PROJECT_ID>` exit_status should eq 0
     ✔  Command: `gsutil ls -p <PROJECT_ID>` stderr should eq ""
     ✔  Command: `gsutil ls -p <PROJECT_ID>` stdout should include "multiple-buckets-mzgy-eu-one"
     ✔  Command: `gsutil ls -p <PROJECT_ID>` stdout should include "<BUCKET-ID-01>"
     ✔  Command: `gsutil bucketpolicyonly get gs://<BUCKET-ID-01>` exit_status should eq 0
     ✔  Command: `gsutil bucketpolicyonly get gs://<BUCKET-ID-01>` stderr should eq ""
     ✔  Command: `gsutil bucketpolicyonly get gs://<BUCKET-ID-01>` stdout should include "Enabled: True"
     ✔  Command: `gsutil bucketpolicyonly get gs://<BUCKET-ID-02>` exit_status should eq 0
     ✔  Command: `gsutil bucketpolicyonly get gs://<BUCKET-ID-02>` stderr should eq ""
     ✔  Command: `gsutil bucketpolicyonly get gs://<BUCKET-ID-02>` stdout should include "Enabled: False"
     ✔  Command: `gsutil label get gs://<BUCKET-ID-01>` exit_status should eq 0
     ✔  Command: `gsutil label get gs://<BUCKET-ID-01>` stderr should eq ""
     ✔  Command: `gsutil label get gs://<BUCKET-ID-01>` bucket_1 has label
     ✔  Command: `gsutil label get gs://<BUCKET-ID-02>` exit_status should eq 0
     ✔  Command: `gsutil label get gs://<BUCKET-ID-02>` stderr should eq ""
     ✔  Command: `gsutil label get gs://<BUCKET-ID-02>` bucket_2 has label
     ✔  Command: `gsutil lifecycle get gs://<BUCKET-ID-01>` should eq "NEARLINE"
     ✔  Command: `gsutil lifecycle get gs://<BUCKET-ID-01>` should eq "SetStorageClass"
     ✔  Command: `gsutil lifecycle get gs://<BUCKET-ID-01>` should eq 10
     ✔  Command: `gsutil lifecycle get gs://<BUCKET-ID-01>` should eq false
     ✔  Command: `gsutil lifecycle get gs://<BUCKET-ID-01>` should eq ["MULTI_REGIONAL", "STANDARD", "DURABLE_REDUCED_AVAILABILITY"]
     ✔  Command: `gsutil lifecycle get gs://<BUCKET-ID-01>` exit_status should eq 0
     ✔  Command: `gsutil lifecycle get gs://<BUCKET-ID-01>` stderr should eq ""
     ✔  Command: `gsutil lifecycle get gs://<BUCKET-ID-02>` should eq "NEARLINE"
     ✔  Command: `gsutil lifecycle get gs://<BUCKET-ID-02>` should eq "SetStorageClass"
     ✔  Command: `gsutil lifecycle get gs://<BUCKET-ID-02>` should eq 10
     ✔  Command: `gsutil lifecycle get gs://<BUCKET-ID-02>` should eq false
     ✔  Command: `gsutil lifecycle get gs://<BUCKET-ID-02>` should eq ["MULTI_REGIONAL", "STANDARD", "DURABLE_REDUCED_AVAILABILITY"]
     ✔  Command: `gsutil lifecycle get gs://<BUCKET-ID-02>` exit_status should eq 0
     ✔  Command: `gsutil lifecycle get gs://<BUCKET-ID-02>` stderr should eq ""


Profile Summary: 1 successful control, 0 control failures, 0 controls skipped
Test Summary: 30 successful, 0 failures, 0 skipped
       Finished verifying <multiple-buckets-default> (0m8.83s).
-----> Kitchen is finished. (0m16.61s)

6. Run kitchen_do destroy <EXAMPLE_NAME> to destroy the example module state.

This step destroys the workspace that you created in the above steps. This step will also destroy the GCS buckets that were created in the project along with the label that you added to the GCS module.

You can view the below output in your terminal.

[root@<CONTAINER_ID> workspace]# kitchen_do destroy multiple-buckets-default
Automatically setting inputs from outputs of test/setup
Found test/source.sh. Using it for additional explicit environment configuration.
Activated service account credentials for: [[e-mail]]
-----> Starting Kitchen (v1.24.0)
-----> Destroying <multiple-buckets-default>...
       Terraform v0.12.12
       + provider.google v3.10.0
       
       Your version of Terraform is out of date! The latest version
       is 0.12.21. You can update by downloading from www.terraform.io/downloads.html
$$$$$$ Running command `terraform init -input=false -lock=true -lock-timeout=0s  -force-copy -backend=true  -get=true -get-plugins=true -verify-plugins=true` in directory /workspace/test/fixtures/mu
ltiple_buckets
       Initializing modules...
       
       Initializing the backend...
       
       Initializing provider plugins...
       
       Terraform has been successfully initialized!
$$$$$$ Running command `terraform workspace select kitchen-terraform-multiple-buckets-default` in directory /workspace/test/fixtures/multiple_buckets
$$$$$$ Running command `terraform destroy -auto-approve -lock=true -lock-timeout=0s -input=false  -parallelism=10 -refresh=true  ` in directory /workspace/test/fixtures/multiple_buckets
       random_string.prefix: Refreshing state... [id=mzgy]
       module.example.module.cloud_storage.google_storage_bucket.buckets[0]: Refreshing state... [id=<BUCKET-ID-01>]
       module.example.module.cloud_storage.google_storage_bucket.buckets[1]: Refreshing state... [id=<BUCKET-ID-02>]
       module.example.module.cloud_storage.google_storage_bucket.buckets[0]: Destroying... [id=<BUCKET-ID-01>]
       module.example.module.cloud_storage.google_storage_bucket.buckets[1]: Destroying... [id=<BUCKET-ID-02>]
       module.example.module.cloud_storage.google_storage_bucket.buckets[0]: Destruction complete after 1s
       module.example.module.cloud_storage.google_storage_bucket.buckets[1]: Destruction complete after 2s
       random_string.prefix: Destroying... [id=mzgy]
       random_string.prefix: Destruction complete after 0s
       
       Destroy complete! Resources: 3 destroyed.
$$$$$$ Running command `terraform workspace select default` in directory /workspace/test/fixtures/multiple_buckets
       Switched to workspace "default".
$$$$$$ Running command `terraform workspace delete kitchen-terraform-multiple-buckets-default` in directory /workspace/test/fixtures/multiple_buckets
       Deleted workspace "kitchen-terraform-multiple-buckets-default"!
       Finished destroying <multiple-buckets-default> (0m6.49s).
-----> Kitchen is finished. (0m8.10s)

9. Generating Documentation for Inputs and Outputs

The Inputs and Outputs tables in the READMEs of the root module, submodules, and example modules are automatically generated based on the variables and outputs of the respective modules. These tables must be refreshed if the module interfaces are changed.

Run:

make generate_docs
# This will generate new Inputs and Outputs tables

Remember: Always run generate_docs to ensure all documentation is complete.

10. Execute lint tests in CFT

A linter is a tool that analyzes source code to flag programming errors, bugs, stylistic errors, and suspicious constructs.

Many of the files in the repository can be linted or formatted to maintain a standard of quality. To ensure quality in CFT, you will use a lint test.

Run:

make docker_test_lint
# This will run all lint tests on your repo

Remember: Always run the lint test because the Cloud Build will fail if the linting is not complete. Run this before committing code to your local repo.

11. Submitting a PR on Github

Now that you have changed your code locally and tested it through the integration tests you would want to publish this code to the master repo.

To make your code available on the master repo, you will need to commit code changes to your branch and push it to the master repository. For your code to be added to the main repo that you forked at the start of the Codelab, you will raise a Pull Request (PR) on the master repo after committing code to your repo.

When you raise a PR, the repo admin will be notified to review the proposed code changes. Additionally, you can also add other users as reviewers to get feedback on your code changes. The PR will trigger a Cloud Build which will run tests on the repo.

Based on your code changes, code reviewers will provide comments on your code and ask for modifications if something needs to be changed based on best practices and documentation. The admin will review your code changes, ensure that your code is compliant with the repo and might again request you to make some changes before merging your code into the master repo.

Execute the following steps to commit code to your forked branch and push code to your forked branch:

1. First step is to add changed files into the local repo.

$ git add main.tf
$ git add README.md
$ git add variables.tf
$ git add examples/multiple-buckets/main.tf
$ git add test/integration/multiple-buckets/controls/gsutil.rb
# The ‘git add' command adds the file in the local repository and 
# stages the file for commit. To unstage a file, use git reset HEAD YOUR-FILE

2. Your files are now staged, next you will commit the changes.

$ git commit -m "First CFT commit"
# This will commit the staged changes and prepares them to be pushed 
# to a remote repository. To remove this commit and modify the file, 
# use 'git reset --soft HEAD~1' and commit and add the file again.

3. Push the committed changes in your local repository to GitHub for creating a Pull Request (PR).

$ git push -u origin master
# Pushes the changes in your local repository up to the remote
# repository you specified as the origin

Your code changes are now ready for a Pull Request!

Execute the following steps to raise a PR to the terraform-google-modules/terraform-google-cloud-storage repo:

1. In your web browser, navigate to the main page of the repo.
2. In the Branch menu, choose your fork that contains your commits.
3. To the right of the "Branch" menu, click "New pull request".

4. Use the base "base" dropdown menu to select the branch you would like to merge your changes into, usually this is the "master" branch, since you have committed code changes to your fork.
5. Enter a title and description for your pull request to describe your code changes. Be as specific as possible while being concise.
6. To create a pull request that is ready for review, click "Create Pull Request".

7. You will see Cloud Build Triggers running which are triggered due to the PR.

You have successfully pushed your first code change to your forked branch and raised your first CFT PR against the master branch!

12. Congratulations

Congratulations, you've successfully added a feature to a CFT module and submitted a PR for review!

You added a feature to a CFT module, tested it locally through an example, and performed tests before committing your code to GitHub. Finally, you raised a PR for review and final merge into CFT.

You now know the important steps to get started with the Cloud Foundation Toolkit.