Google Container Engine makes it easy to run docker containers in the cloud. Google Container Engine uses Kubernetes, an open source container scheduler, to ensure that your cluster is running exactly the way you want it to at all times.

Follow along this lab to learn how to launch a MongoDB Replica set on Google Container Engine using the StatefulSet object.

What you'll learn

What you'll need

How will you use this tutorial?

Read it through only Read it and complete the exercises

How would rate your experience with Google Cloud Platform?

Novice Intermediate Proficient

Self-paced environment setup

If you don't already have a Google Account (Gmail or Google Apps), you must create one. Sign-in to Google Cloud Platform console (console.cloud.google.com) and create a new project:

Remember the project ID, a unique name across all Google Cloud projects (the name above has already been taken and will not work for you, sorry!). It will be referred to later in this codelab as PROJECT_ID.

Next, you'll need to enable billing in the Developers Console in order to use Google Cloud resources like Cloud Datastore and Cloud Storage.

Running through this codelab shouldn't cost you more than a few dollars, but it could be more if you decide to use more resources or if you leave them running (see "cleanup" section at the end of this document).

New users of Google Cloud Platform are eligible for a $300 free trial.

In this section you'll create a Google Container Engine cluster.

Login to Google Cloud Console

Navigate to the the Google Cloud Console from another browser tab/window, to https://console.cloud.google.com. Use the login credential given to you by the lab proctor.

Setup Project Prerequisites

Enable APIs

Search for "Google Compute Engine" in the search box. Click on "Google Compute Engine" in the results list that appears.

Now click "Enable"

Set Compute Zone

Launch Cloud Shell by clicking on the terminal icon in the top toolbar.

Cloud Shell is a browser based terminal to a virtual machine that has the Google Cloud Platform tools installed on it and some additional tools (like editors and compilers) that are handy when you are developing or debugging your cloud application.

We'll be using the gcloud command to create the cluster. First, though, we need to set the compute zone so that the virtual machines in our cluster are created in the correct region. We can do this using gcloud config set compute/zone. Enter the following in Cloud Shell.

gcloud config set compute/zone us-central1-f

You can create a new container cluster with the gcloud command like this:

gcloud container clusters create hello-world

This command creates a new cluster called "hello-world" with three nodes (VMs). You can configure this command with additional flags to change the number of nodes, the default permissions, and other variables. See the documentation for more details.

Launching the cluster may take a bit of time but once it is up you should see output in Cloud Shell that looks like this:

NAME         ZONE           MASTER_VERSION  MASTER_IP        MACHINE_TYPE   NODE_VERSION  NUM_NODES  STATUS
hello-world  us-central1-f  1.5.2           104.197.119.168  n1-standard-1  1.5.2         3          RUNNING

To set up the MongoDB replica set, you need three things: A StorageClass, a Headless Service, and a StatefulSet.

You can clone the example configuration files from GitHub.

git clone https://github.com/thesandlord/mongo-k8s-sidecar.git
cd ./mongo-k8s-sidecar/example/StatefulSet/

Create the storage class

The storage class tells Kubernetes what kind of storage to use for the database nodes. On Google Cloud Platform, your storage choices are SSDs and hard disks.

The configuration for the StorageClass looks like this:

kind: StorageClass
apiVersion: storage.k8s.io/v1beta1
metadata:
  name: fast
provisioner: kubernetes.io/gce-pd
parameters:
  type: pd-ssd

This configuration creates a new StorageClass called "fast" that is backed by SSD volumes. The StatefulSet can now request a volume, and the StorageClass will automatically create it.

Deploy this StorageClass:

kubectl apply -f googlecloud_ssd.yaml

Headless Service

The configuration for the Headless Service looks like this:

apiVersion: v1
kind: Service
metadata:
 name: mongo
 labels:
   name: mongo
spec:
 ports:
 - port: 27017
   targetPort: 27017
 clusterIP: None
 selector:
   role: mongo

You can tell this is a Headless Service because the clusterIP is set to "None." When combined with StatefulSets, they can give you unique DNS addresses that let you directly access the pods. This is perfect for creating MongoDB replica sets, because our app needs to connect to all of the MongoDB nodes individually.

We will deploy the headless service in the next step along with the StatefulSet

StatefulSet

The configuration for the StatefulSet looks like this:

apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
 name: mongo
spec:
 serviceName: "mongo"
 replicas: 3
 template:
   metadata:
     labels:
       role: mongo
       environment: test
   spec:
     terminationGracePeriodSeconds: 10
     containers:
       - name: mongo
         image: mongo
         command:
           - mongod
           - "--replSet"
           - rs0
           - "--smallfiles"
           - "--noprealloc"
         ports:
           - containerPort: 27017
         volumeMounts:
           - name: mongo-persistent-storage
             mountPath: /data/db
       - name: mongo-sidecar
         image: cvallance/mongo-k8s-sidecar
         env:
           - name: MONGO_SIDECAR_POD_LABELS
             value: "role=mongo,environment=test"
 volumeClaimTemplates:
 - metadata:
     name: mongo-persistent-storage
     annotations:
       volume.beta.kubernetes.io/storage-class: "fast"
   spec:
     accessModes: [ "ReadWriteOnce" ]
     resources:
       requests:
         storage: 100Gi

It's a little long, but fairly straightforward.

The first second describes the StatefulSet object. Then, we move into the Metadata section, where you can specify labels and the number of replicas.

Next comes the pod spec. The terminationGracePeriodSeconds is used to gracefully shutdown the pod when you scale down the number of replicas, which is important for databases! Then the configurations for the two containers is shown. The first one runs MongoDB with command line flags that configure the replica set name. It also mounts the persistent storage volume to /data/db, the location where MongoDB saves its data. The second container runs the sidecar. This "sidecar" container will configure the MongoDB replica set automatically. A "sidecar" is a helper container which helps the main container do its work.

Finally, there is the volumeClaimTemplates. This is what talks to the StorageClass we created before to provision the volume. It will provision a 100 GB disk for each MongoDB replica.

You can deploy both the Headless Service and the StatefulSet with this command:

kubectl apply -f mongo-statefulset.yaml

Now that we have a cluster running and our Replica Set deployed, we can connect to it.

Wait for the MongoDB Replica Set to be fully deployed

Kubernetes StatefulSets will deploy each pod sequentially. It will wait for the MongoDB Replica Set member to fully boot up and create the backing disk before starting the next member. In a few minutes, you should see the "Desired" and "Current" count to be equal:

kubectl get statefulset

You should eventually get back a result that looks something like:

NAME      DESIRED   CURRENT   AGE
mongo     3         3         3m

Viewing the MongoDB replica set

At this point, you should have three pods created in your cluster. These correspond to the three nodes in your MongoDB replica set. You can see them with this command:

kubectl get pods

You should get back a result that looks something like the following. If not, wait for all three members to be created:

NAME        READY     STATUS    RESTARTS   AGE
mongo-0     2/2       Running   0          3m
mongo-1     2/2       Running   0          3m
mongo-2     2/2       Running   0          3m

Let's connect to the first Replica Set member.

kubectl exec -ti mongo-0 mongo

You should now have a REPL connected to the MongoDB Replica Set.

Print the replica set configuration with the rs.conf() command:

rs.conf()

At this point, you can see all three members of the Replica Set in the configuration, and the Replica Set is named "rs0."

rs0:PRIMARY> rs.conf()
{
    "_id": "rs0",
    "version": 5,
    "protocolVersion": NumberLong(1),
    "members": [
        {
            "_id": 0,
            "host": "10.160.2.5:27017",
            "arbiterOnly": false,
            "buildIndexes": true,
            "hidden": false,
            "priority": 1,
            "tags": {
                
            },
            "slaveDelay": NumberLong(0),
            "votes": 1
        },
        {
            "_id": 1,
            "host": "10.160.0.6:27017",
            "arbiterOnly": false,
            "buildIndexes": true,
            "hidden": false,
            "priority": 1,
            "tags": {
                
            },
            "slaveDelay": NumberLong(0),
            "votes": 1
        },
        {
            "_id": 2,
            "host": "10.160.2.6:27017",
            "arbiterOnly": false,
            "buildIndexes": true,
            "hidden": false,
            "priority": 1,
            "tags": {
                
            },
            "slaveDelay": NumberLong(0),
            "votes": 1
        }
    ],
    "settings": {
        "chainingAllowed": true,
        "heartbeatIntervalMillis": 2000,
        "heartbeatTimeoutSecs": 10,
        "electionTimeoutMillis": 10000,
        "catchUpTimeoutMillis": 2000,
        "getLastErrorModes": {
            
        },
        "getLastErrorDefaults": {
            "w": 1,
            "wtimeout": 0
        },
        "replicaSetId": ObjectId("58ace74774ebb9712eb31d3f")
    }
}

Type "exit" and press enter to quit the REPL.

A big advantage of Kubernetes and StatefulSets is that you can scale up and down the number of MongoDB Replicas with a single command!

To scale up the number of Replica Set members from 3 to 5, run this command:

kubectl scale --replicas=5 statefulset mongo

In a few minutes, you will see that there are 5 MongoDB pods.

kubectl get pods

You should get back a result that looks something like:

NAME      READY     STATUS    RESTARTS   AGE
mongo-0   2/2       Running   0          41m
mongo-1   2/2       Running   0          39m
mongo-2   2/2       Running   0          37m
mongo-3   2/2       Running   0          4m
mongo-4   2/2       Running   0          2m

To scale down the number of Replica Set members from 5 back to 3, run this command:

kubectl scale --replicas=3 statefulset mongo

In a few seconds, you will see that there are 3 MongoDB pods.

kubectl get pods

You should get back a result that looks something like:

NAME      READY     STATUS    RESTARTS   AGE
mongo-0   2/2       Running   0          41m
mongo-1   2/2       Running   0          39m
mongo-2   2/2       Running   0          37m

Each pod in a StatefulSet backed by a Headless Service will have a stable DNS name. The template follows this format: <pod-name>.<service-name>

This means the DNS names for the MongoDB replica set are:

mongo-0.mongo
mongo-1.mongo
mongo-2.mongo

You can use these names directly in the connection string URI of your app.

In this case, the connection string URI would be:

"mongodb://mongo-0.mongo,mongo-1.mongo,mongo-2.mongo:27017/dbname_?"

Using the database is outside the scope of this codelab.

To clean up the deployed resources, delete the StatefulSet, Headless Service, and the provisioned volumes.

Delete the StatefulSet:

kubectl delete statefulset mongo

Delete the Service:

kubectl delete svc mongo

Delete the Volumes:

kubectl delete pvc -l role=mongo

Finally, you can delete the test cluster:

gcloud container clusters delete "hello-world"

Google Container Engine and Kubernetes provide a powerful and flexible way to run containers on Google Cloud Platform. StatefulSets let you run stateful workloads like databases on Kubernetes.

What we've covered

Next Steps