Continuous deployment to Google Kubernetes Engine (GKE) with Cloud Build

1. Overview

In this lab, you'll learn to set up a continuous delivery pipeline for GKE with Cloud Build. This lab highlights how to trigger Cloud Build jobs for different git events as well as a simple pattern for automated canary releases in GKE.

You'll complete the following steps:

  • Create the GKE Application
  • Automate deployments for git branches
  • Automate deployments for git main branch
  • Automate deployments for git tags

2. Before you begin

For this reference guide, you need a Google Cloud project. You can create a new one, or select a project you already created:

  1. Select or create a Google Cloud project.

GO TO THE PROJECT SELECTOR PAGE

  1. Enable billing for your project.

ENABLE BILLING

3. Preparing your environment

  1. Create environment variables to use throughout this tutorial:
    export PROJECT_ID=$(gcloud config get-value project)
    export PROJECT_NUMBER=$(gcloud projects describe $PROJECT_ID --format='value(projectNumber)')
    
    export ZONE=us-central1-b
    export CLUSTER=gke-progression-cluster
    export APP_NAME=myapp
    
  2. Enable the following APIs:
    • Resource Manager
    • GKE
    • Cloud Source Repositories
    • Cloud Build
    • Container Registry
    gcloud services enable \
        cloudresourcemanager.googleapis.com \
        container.googleapis.com \
        sourcerepo.googleapis.com \
        cloudbuild.googleapis.com \
        containerregistry.googleapis.com \
        --async
    
  3. Clone the sample source and switch to the lab directory:
    git clone https://github.com/GoogleCloudPlatform/software-delivery-workshop.git gke-progression
    
    cd gke-progression/labs/gke-progression
    rm -rf ../../.git
    
  4. Replace placeholder values in the sample repository with your PROJECT_ID:In this step you create instances of the various config files unique to your current environment.To review an example of the templates being updated, run the following command.
    cat k8s/deployments/dev/frontend-dev.yaml.tmpl
    
    Perform the variable substitution by executing the followign command.
    for template in $(find . -name '*.tmpl'); do envsubst '${PROJECT_ID} ${ZONE} ${CLUSTER} ${APP_NAME}' < ${template} > ${template%.*}; done
    
    To review an example of the file after the substitution, run the following command.
    cat k8s/deployments/dev/frontend-dev.yaml
    
  5. If you haven't used Git in Cloud Shell previously, set the user.name and user.email values that you want to use:
    git config --global user.email "YOUR_EMAIL_ADDRESS"
    git config --global user.name "YOUR_USERNAME"
    
  6. Store the code from the sample repository in Cloud Source Repositories:
    gcloud source repos create gke-progression
    git init
    git config credential.helper gcloud.sh
    git remote add gcp https://source.developers.google.com/p/$PROJECT_ID/r/gke-progression
    git branch -m main
    git add . && git commit -m "initial commit"
    git push gcp main
    
  7. Create your GKE cluster.
    gcloud container clusters create ${CLUSTER} \
        --project=${PROJECT_ID} \
        --zone=${ZONE}
    
  8. Give Cloud Build rights to your cluster.Cloud Build will be deploying the application to your GKE Cluster and will need rights to do so.
    gcloud projects add-iam-policy-binding ${PROJECT_ID} \
        --member=serviceAccount:${PROJECT_NUMBER}@cloudbuild.gserviceaccount.com \
        --role=roles/container.developer
    

Your environment is ready!

4. Creating your GKE Application

In this section, you build and deploy the initial production application that you use throughout this tutorial.

  1. Build the application with Cloud Build:
    gcloud builds submit --tag gcr.io/$PROJECT_ID/$APP_NAME:1.0.0 src/
    
  2. Manually deploy to Canary and Production environments:Create the production and canary deployments and services using the kubectl apply commands.
    kubectl create ns production
    kubectl apply -f k8s/deployments/prod -n production
    kubectl apply -f k8s/deployments/canary -n production
    kubectl apply -f k8s/services -n production
    
    The service deployed here will route traffic to both the canary and prod deployments.
  3. Review number of running podsConfirm that you have four Pods running for the frontend, including three for production traffic and one for canary releases. That means that changes to your canary release will only affect 1 out of 4 (25%) of users.
    kubectl get pods -n production -l app=$APP_NAME -l role=frontend
    
  4. Retrieve the external IP address for the production services.
    kubectl get service $APP_NAME -n production
    
    Once the load balancer returns the IP address continue to the next step
  5. Store the external IP for later use.
    export PRODUCTION_IP=$(kubectl get -o jsonpath="{.status.loadBalancer.ingress[0].ip}"  --namespace=production services $APP_NAME)
    
  6. Review the application Check the version output of the service. It should read Hello World v1.0
    curl http://$PRODUCTION_IP
    

Congratulations! You deployed the sample app! Next, you'll set up a triggers for continuously deploying your changes.

5. Automating deployments for git branches

In this section you will set up a trigger that will execute a Cloudbuild job on commit of any branch other than main. The Cloud Build file used here will automatically create a namespace and deployment for any existing or new branches, allowing developers to preview their code before integration with the main branch.

  1. Set up the trigger:The key component of this trigger is the use of the branchName parameter to match main and the invertRegex parameter which is set to true and alters the branchName pattern to match anything that is not main. For your reference you can find the following lines in build/branch-trigger.json.
      "branchName": "main",
      "invertRegex": true
    
    Additionally the last few lines of the Cloud Build file used with this trigger create a namespace named after the branch that triggered the job, then deploys the application and service within the new namespace. For your reference you can find the following lines in build/branch-cloudbuild.yaml
      kubectl get ns ${BRANCH_NAME} || kubectl create ns ${BRANCH_NAME}
      kubectl apply --namespace ${BRANCH_NAME} --recursive -f k8s/deployments/dev
      kubectl apply --namespace ${BRANCH_NAME} --recursive -f k8s/services
    
    Now that you understand the mechanisms in use, create the trigger with the gcloud command below.
    gcloud beta builds triggers create cloud-source-repositories \
      --trigger-config build/branch-trigger.json
    
  2. To review the trigger, go to the Cloud Build Triggers page in the Console.Go to Triggers
  3. Create a new branch:
    git checkout -b new-feature-1
    
  4. Modify the code to indicate v1.1Edit src/app.py and change the response from 1.0 to 1.1
    @app.route('/')
    def hello_world():
        return 'Hello World v1.1'
    
  5. Commit the change and push to the remote repository:
    git add . && git commit -m "updated" && git push gcp new-feature-1
    
  6. To review the build in progress, go to the Cloud Build History page in the Console.Go to BuildsOnce the build completes continue to the next step
  7. Retrieve the external IP address for the newly deployed branch service.
    kubectl get service $APP_NAME -n new-feature-1
    
    Once the load balancer returns the IP address continue to the next step
  8. Store the external IP for later use.
    export BRANCH_IP=$(kubectl get -o jsonpath="{.status.loadBalancer.ingress[0].ip}"  --namespace=new-feature-1 services $APP_NAME)
    
  9. Review the applicationCheck the version output of the service. It should read Hello World v1.0
    curl http://$BRANCH_IP
    

6. Automating deployments for git main branch

Before code is released to production, it's common to release code to a small subset of live traffic before migrating all traffic to the new code base.

In this section, you implement a trigger that is activated when code is committed to the main branch. The trigger deploys the canary deployment which receives 25% of all live traffic to the new revision.

  1. Set up the trigger for the main branch:
    gcloud beta builds triggers create cloud-source-repositories \
      --trigger-config build/main-trigger.json
    
  2. To review the new trigger, go to the Cloud Build Triggers page in the Console.Go to Triggers
  3. Merge the branch to the main line and push to the remote repository:
    git checkout main
    git merge new-feature-1
    git push gcp main
    
  4. To review the build in progress, go to the Cloud Build History page in the Console.Go to BuildsOnce the build has completed continue to the next step
  5. Review multiple responses from the serverRun the following command and note that approximately 25% of the responses are showing the new response of Hello World v1.1
    while true; do curl -w "\n" http://$PRODUCTION_IP; sleep 1;  done
    
    When you're ready to continue press Ctrl+c to exit out of the loop.

7. Automating deployments for git tags

After the canary deployment is validated with a small subset of traffic, you release the deployment to the remainder of the live traffic.

In this section, you set up a trigger that is activated when you create a tag in the repository. The trigger labels the image with the appropriate tag then deploys the updates to prod ensuring 100% of traffic is accessing the tagged image.

  1. Set up the tag trigger:
    gcloud beta builds triggers create cloud-source-repositories \
      --trigger-config build/tag-trigger.json
    
  2. To review the new trigger, go to the Cloud Build Triggers page in the Console.Go to Triggers
  3. Create a new tag and push to the remote repository:
    git tag 1.1
    git push gcp 1.1
    
  4. To review the build in progress, go to the Cloud Build History page in the Console.Go to Builds
  5. Review multiple responses from the serverRun the following command and note that 100% of the responses are showing the new response of Hello World v1.1This may take a moment as the new pods are deployed and health checked within GKE
    while true; do curl -w "\n" http://$PRODUCTION_IP; sleep 1;  done
    
    When you're ready to continue press Ctrl+c to exit out of the loop.Congratulations! You created CI/CD triggers in Cloud Build for branches and tags to deploy your apps to GKE.

8. Cleanup

Delete the project

  1. In the Cloud Console, go to the Manage resources page.
  2. In the project list, select the project that you want to delete, and then click Delete.
  3. In the dialog, type the project ID, and then click Shut down to delete the project.