Continuous deployment to Google Kubernetes Engine (GKE) with Cloud Build

Continuous deployment to Google Kubernetes Engine (GKE) with Cloud Build

About this codelab

Last updated Feb 28, 2022

Written by crgrant

1. Overview

In this lab, you'll learn to set up a continuous delivery pipeline for GKE with Cloud Build. This lab highlights how to trigger Cloud Build jobs for different git events as well as a simple pattern for automated canary releases in GKE.

You'll complete the following steps:

• Create the GKE Application
• Automate deployments for git branches
• Automate deployments for git main branch
• Automate deployments for git tags

2. Before you begin

For this reference guide, you need a Google Cloud project. You can create a new one, or select a project you already created:

1. Select or create a Google Cloud project.

GO TO THE PROJECT SELECTOR PAGE

1. Enable billing for your project.

ENABLE BILLING

3. Preparing your environment

1. Create environment variables to use throughout this tutorial:

   export PROJECT_ID=$(gcloud config get-value project)
   export PROJECT_NUMBER=$(gcloud projects describe $PROJECT_ID --format='value(projectNumber)')
   
   export ZONE=us-central1-b
   export CLUSTER=gke-progression-cluster
   export APP_NAME=myapp
   

2. Enable the following APIs:
   • Resource Manager
   • GKE
   • Cloud Source Repositories
   • Cloud Build
   • Container Registry

   gcloud services enable \
       cloudresourcemanager.googleapis.com \
       container.googleapis.com \
       sourcerepo.googleapis.com \
       cloudbuild.googleapis.com \
       containerregistry.googleapis.com \
       --async
   

3. Clone the sample source and switch to the lab directory:

   git clone https://github.com/GoogleCloudPlatform/software-delivery-workshop.git gke-progression
   
   cd gke-progression/labs/gke-progression
   rm -rf ../../.git
   

4. Replace placeholder values in the sample repository with your PROJECT_ID:In this step you create instances of the various config files unique to your current environment.To review an example of the templates being updated, run the following command.

   cat k8s/deployments/dev/frontend-dev.yaml.tmpl
   

   Perform the variable substitution by executing the followign command.

   for template in $(find . -name '*.tmpl'); do envsubst '${PROJECT_ID} ${ZONE} ${CLUSTER} ${APP_NAME}' < ${template} > ${template%.*}; done
   

   To review an example of the file after the substitution, run the following command.

   cat k8s/deployments/dev/frontend-dev.yaml
   

5. If you haven't used Git in Cloud Shell previously, set the user.name and user.email values that you want to use:

   git config --global user.email "YOUR_EMAIL_ADDRESS"
   git config --global user.name "YOUR_USERNAME"
   

6. Store the code from the sample repository in Cloud Source Repositories:

   gcloud source repos create gke-progression
   git init
   git config credential.helper gcloud.sh
   git remote add gcp https://source.developers.google.com/p/$PROJECT_ID/r/gke-progression
   git branch -m main
   git add . && git commit -m "initial commit"
   git push gcp main
   

7. Create your GKE cluster.

   gcloud container clusters create ${CLUSTER} \
       --project=${PROJECT_ID} \
       --zone=${ZONE}
   

8. Give Cloud Build rights to your cluster.Cloud Build will be deploying the application to your GKE Cluster and will need rights to do so.

   gcloud projects add-iam-policy-binding ${PROJECT_ID} \
       --member=serviceAccount:${PROJECT_NUMBER}@cloudbuild.gserviceaccount.com \
       --role=roles/container.developer
   

Your environment is ready!

4. Creating your GKE Application

In this section, you build and deploy the initial production application that you use throughout this tutorial.

1. Build the application with Cloud Build:

   gcloud builds submit --tag gcr.io/$PROJECT_ID/$APP_NAME:1.0.0 src/
   

2. Manually deploy to Canary and Production environments:Create the production and canary deployments and services using the kubectl apply commands.

   kubectl create ns production
   kubectl apply -f k8s/deployments/prod -n production
   kubectl apply -f k8s/deployments/canary -n production
   kubectl apply -f k8s/services -n production
   

   The service deployed here will route traffic to both the canary and prod deployments.
3. Review number of running podsConfirm that you have four Pods running for the frontend, including three for production traffic and one for canary releases. That means that changes to your canary release will only affect 1 out of 4 (25%) of users.

   kubectl get pods -n production -l app=$APP_NAME -l role=frontend
   

4. Retrieve the external IP address for the production services.

   Note: It can take several minutes before you see the load balancer external IP address.

   kubectl get service $APP_NAME -n production
   

   Once the load balancer returns the IP address continue to the next step
5. Store the external IP for later use.

   export PRODUCTION_IP=$(kubectl get -o jsonpath="{.status.loadBalancer.ingress[0].ip}"  --namespace=production services $APP_NAME)
   

6. Review the application Check the version output of the service. It should read Hello World v1.0

   curl http://$PRODUCTION_IP
   

Congratulations! You deployed the sample app! Next, you'll set up a triggers for continuously deploying your changes.

5. Automating deployments for git branches

In this section you will set up a trigger that will execute a Cloudbuild job on commit of any branch other than main. The Cloud Build file used here will automatically create a namespace and deployment for any existing or new branches, allowing developers to preview their code before integration with the main branch.

1. Set up the trigger:The key component of this trigger is the use of the branchName parameter to match main and the invertRegex parameter which is set to true and alters the branchName pattern to match anything that is not main. For your reference you can find the following lines in build/branch-trigger.json.

     "branchName": "main",
     "invertRegex": true
   

   Additionally the last few lines of the Cloud Build file used with this trigger create a namespace named after the branch that triggered the job, then deploys the application and service within the new namespace. For your reference you can find the following lines in build/branch-cloudbuild.yaml

     kubectl get ns ${BRANCH_NAME} || kubectl create ns ${BRANCH_NAME}
     kubectl apply --namespace ${BRANCH_NAME} --recursive -f k8s/deployments/dev
     kubectl apply --namespace ${BRANCH_NAME} --recursive -f k8s/services
   

   Now that you understand the mechanisms in use, create the trigger with the gcloud command below.

   gcloud beta builds triggers create cloud-source-repositories \
     --trigger-config build/branch-trigger.json
   

2. To review the trigger, go to the Cloud Build Triggers page in the Console.Go to Triggers
3. Create a new branch:

   git checkout -b new-feature-1
   

4. Modify the code to indicate v1.1Edit src/app.py and change the response from 1.0 to 1.1

   @app.route('/')
   def hello_world():
       return 'Hello World v1.1'
   

5. Commit the change and push to the remote repository:

   git add . && git commit -m "updated" && git push gcp new-feature-1
   

6. To review the build in progress, go to the Cloud Build History page in the Console.Go to BuildsOnce the build completes continue to the next step
7. Retrieve the external IP address for the newly deployed branch service.

   Note: It can take several minutes before you see the load balancer external IP address.

   kubectl get service $APP_NAME -n new-feature-1
   

   Once the load balancer returns the IP address continue to the next step
8. Store the external IP for later use.

   export BRANCH_IP=$(kubectl get -o jsonpath="{.status.loadBalancer.ingress[0].ip}"  --namespace=new-feature-1 services $APP_NAME)
   

9. Review the applicationCheck the version output of the service. It should read Hello World v1.0

   curl http://$BRANCH_IP
   

6. Automating deployments for git main branch

Before code is released to production, it's common to release code to a small subset of live traffic before migrating all traffic to the new code base.

In this section, you implement a trigger that is activated when code is committed to the main branch. The trigger deploys the canary deployment which receives 25% of all live traffic to the new revision.

1. Set up the trigger for the main branch:

   gcloud beta builds triggers create cloud-source-repositories \
     --trigger-config build/main-trigger.json
   

2. To review the new trigger, go to the Cloud Build Triggers page in the Console.Go to Triggers
3. Merge the branch to the main line and push to the remote repository:

   git checkout main
   git merge new-feature-1
   git push gcp main
   

4. To review the build in progress, go to the Cloud Build History page in the Console.Go to BuildsOnce the build has completed continue to the next step
5. Review multiple responses from the serverRun the following command and note that approximately 25% of the responses are showing the new response of Hello World v1.1

   while true; do curl -w "\n" http://$PRODUCTION_IP; sleep 1;  done
   

   When you're ready to continue press Ctrl+c to exit out of the loop.

7. Automating deployments for git tags

After the canary deployment is validated with a small subset of traffic, you release the deployment to the remainder of the live traffic.

In this section, you set up a trigger that is activated when you create a tag in the repository. The trigger labels the image with the appropriate tag then deploys the updates to prod ensuring 100% of traffic is accessing the tagged image.

1. Set up the tag trigger:

   gcloud beta builds triggers create cloud-source-repositories \
     --trigger-config build/tag-trigger.json
   

2. To review the new trigger, go to the Cloud Build Triggers page in the Console.Go to Triggers
3. Create a new tag and push to the remote repository:

   git tag 1.1
   git push gcp 1.1
   

4. To review the build in progress, go to the Cloud Build History page in the Console.Go to Builds
5. Review multiple responses from the serverRun the following command and note that 100% of the responses are showing the new response of Hello World v1.1This may take a moment as the new pods are deployed and health checked within GKE

   while true; do curl -w "\n" http://$PRODUCTION_IP; sleep 1;  done
   

   When you're ready to continue press Ctrl+c to exit out of the loop.Congratulations! You created CI/CD triggers in Cloud Build for branches and tags to deploy your apps to GKE.

8. Cleanup

Delete the project

Caution: Deleting a project has the following effects: Everything in the project is deleted. If you used an existing project for this tutorial, when you delete it, you also delete any other work you've done in the project.

1. In the Cloud Console, go to the Manage resources page.
2. In the project list, select the project that you want to delete, and then click Delete.
3. In the dialog, type the project ID, and then click Shut down to delete the project.