In this lab, you create a Compute Engine snapshot of the boot persistent disk associated with your jenkins-master instance. You create a solid state drive (SSD) boot persistent disk from the snapshot in an alternate zone and attach it to a test instance to verify the proper restoration from the snapshot. You also create, attach, and snapshot a data disk to jenkins-master.

What you need

To complete this lab, you need:

To complete all previous labs

Access to a supported Internet browser:

What you learn

In this lab, you:

You create a backup of your Jenkins master server boot disk using Compute Engine Snapshots. You restore this backup to a different type of disk in a different zone. You test this restored disk by creating a new instance, attaching the restored disk to the instance, booting the instance, and finally logging into the restored Jenkins web user interface.

Compute Engine Snapshots provide a mechanism to create one or a series of persistent disk backups, each at a specific point-in-time. Snapshots are stored as differential captures of the actual data on a persistent disk, using storage space efficiently.

Before a snapshot is created, the persistent disk must be in the exact state desired for the backup and all updates to the disk must be halted. This means that a snapshot should not be captured while data is being written to the disk by an application or a DBMS. It also means that all operating system buffers must be flushed to disk. The steps required to bring a persistent disk to this state will vary depending on the operating system and file system technology used by the instance. A reliable way to ensure that a disk is ready for a snapshot is to unmount the file system. In this lab, you will take an additional step and stop the associated instance before creating a snapshot.

The environment and workflow for this lab relating to Jenkins are illustrated in the following diagram.

Google strongly recommends segregating application data from operating system and application code stored in a file system on different persistent disks. This can be done by creating a separate persistent disk for data and mounting it in the file system on a specific mount point. One advantage of doing this is that it simplifies taking snapshots of a running system.

In this training scenario, you take a backup of the complete boot disk that includes the installed software, configuration data, and application data. Jenkins relies on a relatively small amount of application data and in a later lab you configure the software to backup this data to Google Cloud Storage Nearline. At the end of this lab, you run through the process of creating a snapshot of a data disk without shutting down an instance.

Unlike Compute Engine instances and persistent disks, snapshots are a global resource. Snapshots can be used to restore data to a disk in any zone.

Snapshots are versatile. They can also be used for progressive differential backups, migrating data across zones and regions, migrating data to a different type of disk resource, as well as transferring data from a disk of one allocated size to another disk with a larger allocated size. The last use case requires that the associated file system also be resized by the operating system. Some operating systems support automatic resizing of the associated root file systems. In other cases, additional manual partitioning of disks will be required. Check the documentation for further details.

Also note, that in addition to manually moving Compute Engine instances between zones as demonstrated in this lab, it is possible to largely automate moving instances across zones using either the Cloud SDK or API. Refer to the documentation for further details.

Stop the Jenkins master instance to ensure that all data is flushed to the persistent disk and all writes are halted. Create a snapshot of the boot persistent disk of the jenkins-master instance before starting the Jenkins master once again.

Step 1

Open the Cloud Platform Console, and if necessary, select the Build project.

Step 2

Click Activate Cloud Shell.

Step 3

Type the following command to load the values of all variables for this course into your shell session.

source ~/cpo200/config

Step 4

Type the Cloud SDK command to stop the jenkins-master instance.

gcloud compute instances \
stop jenkins-master \
--zone $CPO200_ZONE

Step 5

Type the following command to check the status of the jenkins-master instance.

gcloud compute instances \

describe jenkins-master \

--zone $CPO200_ZONE | grep status

The output confirms the instance is now terminated:

status: TERMINATED

Step 6

Type the following command to take a snapshot of the jenkins-master disk resource.

gcloud compute disks \
snapshot jenkins-master \
--snapshot-names jenkins-snapshot-1 \
--zone $CPO200_ZONE

The output confirms that the snapshot resource was created.

Step 7

Type the following command to view the details for your new snapshot.

gcloud compute snapshots \
describe jenkins-snapshot-1 \
--format yaml

The output displays the details of the new snapshot in YAML format. Notice that the source disk, marked in bold, that the snapshot is based on is displayed:

creationTimestamp: '<###>'

diskSizeGb: '10'

id: '<###>'

kind: compute#snapshot

name: jenkins-snapshot-1

selfLink: https://www.googleapis.com/compute/v1/projects/<###>

sourceDisk: https://www.googleapis.com/compute/v1/projects/<###>

sourceDiskId: '<###>'

status: READY

storageBytes: '1126255131'

storageBytesStatus: UP_TO_DATE

Step 8

Type the following command to start the jenkins-master instance.

gcloud compute instances \
start jenkins-master \
--zone $CPO200_ZONE

The output confirms that the jenkins-master instance resource was updated.

Step 9

Type the following command to check the status of the jenkins-master instance again.

gcloud compute instances \
describe jenkins-master \
--zone $CPO200_ZONE | grep status

The output confirms the instance is now running:

status: RUNNING

Step 10

Return to the Cloud Platform Console and click Products & services > Compute Engine.

Step 11

Test and verify that the Jenkins web-based user interface is responding to requests by visiting the IP address for your Jenkins master server.

Click the External IP address for the jenkins-master instance.

A new browser tab opens and the Jenkins user interface loads.

Step 12

Close the Jenkins browser tab and leave the Cloud Platform Console and Cloud Shell windows open.

Restore the snapshot of the Jenkins master to a new SSD disk in a different zone from the original disk.

Step 1

In the next step, you select an alternative zone to restore the snapshot to. Type the Cloud SDK command to list the available Compute Engine zones and make a note of possible alternatives to your default zone.

If you are not sure of the syntax, you can find the solution at the of the lab.

Step 2

Pick a zone other than your default zone; you use this to demonstrate moving disks between zones using snapshots.

Type the following command to assign your chosen zone to a temporary shell variable for use in this lab.

ALT_ZONE=<alternative-zone>

There is no output for this command.

Step 3

Type the following command to create an SSD persistent disk resource based on your snapshot.

gcloud compute disks \
create jenkins-restore \
--source-snapshot jenkins-snapshot-1 \
--type pd-ssd \
--zone $ALT_ZONE

The output confirms that the disk resource is created and displays a summary of the properties:

NAME

ZONE

SIZE_GB

TYPE

STATUS

jenkins-restore

<alternative-zone>

10

pd-ssd

READY

In this step, you observe how snapshots can be used to migrate an instance from one zone to another as well as change the type of disk. Can you identify any other potential uses for snapshots when managing the Jenkins master?

If you are not sure of the answer, you can find the solution at the end of the lab.

Step 4

Type the following command to create a Compute Engine instance and to attach the jenkins-restore SSD persistent disk you created in the previous step. You need to edit the command to include the same zone you used in the previous command.

gcloud compute instances \
create jenkins-restore \
--subnet build-subnet \
--tags ssh,http \
--disk name=jenkins-restore,\
device-name=jenkins-restore,\
mode=rw,boot=yes \
--zone $ALT_ZONE

The output displays a summary of the details for the new instance, including the external IP address. Make a temporary note of this address as you will use it in Step 5.

Step 5

Type the following command to compare the details of both persistent disk resources you are currently using in the Jenkins project.

gcloud compute disks list

The output displays two disks in two zones and with two different types:

NAME

ZONE

SIZE_GB

TYPE

STATUS

jenkins-restore

<alternative-zone>

10

pd-ssd

READY

jenkins-master

<your-default-zone>

10

pd-standard

READY

Step 6

Switch back to the Cloud Platform Console and click the External IP address link corresponding to the jenkins-restore instance.

It may take a minute or two for the operating system to boot, for the Jenkins service to load, and for the user interface to become accessible. Verify that Jenkins loads as expected.

Create a new disk and attach to the Jenkins master. Copy data to the disk, synchronize the file system with the disk and make a snapshot without stopping the instance. To verify, create a new disk from the snapshot, attach to the Jenkins master and run a diff command.

Step 1

Return to Cloud Shell and type the Cloud SDK command to create a new disk.

gcloud compute disks \
create data-01 \
--size 200GB \
--zone $CPO200_ZONE

The output confirms the creation of the disk:

NAME ZONE SIZE_GB TYPE STATUS

data-01 <your-default-zone> 200 pd-standard READY

Step 2

Type the following command to attach the new disk to jenkins-master:

gcloud compute instances \
attach-disk jenkins-master \
--disk data-01 \
--device-name disk01 \
--zone $CPO200_ZONE

The output confirms that jenkins-master has been updated:

Updated [.../jenkins-master].

Step 3

Return to the Cloud Platform Console and click the SSH button corresponding to the jenkins-master instance.

This opens an SSH connection to the instance in a new window.

Step 4

Verify the attachment of the disk by typing the following command.

Instance: jenkins-master

ls -d /dev/disk/by-id/* | grep disk01

The output shows the operating system has a reference to the newly attached disk.

/dev/disk/by-id/google-disk01

...

Step 5

Create two mount locations for use with the new disk and for use with a copy you will make later for verification.

Instance: jenkins-master

sudo mkdir -p /mnt/disk01 /mnt/disk02

There is no output from this command.

Step 6

Type the following command to format the new disk.

Instance: jenkins-master

sudo mkfs.ext4 -F -E \
lazy_itable_init=0,lazy_journal_init=0,discard \
/dev/disk/by-id/google-disk01

The output prints the progress of the various operations ending with the following messages:

Allocating group tables: done

Writing inode tables: done

Creating journal (32768 blocks): done

Writing superblocks and filesystem accounting information: done

Step 7

Type the following command to mount the new filesystem.

Instance: jenkins-master

sudo mount -o discard,defaults \
/dev/disk/by-id/google-disk01 \
/mnt/disk01

The command produces no output.

Step 8

Confirm that this is a new disk by typing the following command.

Instance: jenkins-master

ls /mnt/disk01

and observing

lost+found

displayed.

Step 9

Copy data to the new disk from the /home directory by typing the following command.

Instance: jenkins-master

sudo rsync -r /home /mnt/disk01

There is no output from this command.

Step 10

Type the following command to synchronize the file system cache with the disks.

Instance: jenkins-master

sync

There is no output from this command.

Step 11

Type the following command to exit the jenkins-master connection:

Instance: jenkins-master

exit

Step 12

Return to Cloud Shell and type the following command to create a snapshot of the new disk while it remains attached to the running jenkins-master instance.

gcloud compute disks \
snapshot data-01 \
--snapshot-names data-01-snap \
--zone $CPO200_ZONE

The output confirms the creation of the snapshot:

Created [.../data-01-snap].

Step 13

Type the following command to create a new disk from the snapshot.

gcloud compute disks \
create data-02 \
--source-snapshot data-01-snap \
--zone $CPO200_ZONE

The output confirms that the resource has been created and summarizes the details of the disk, including the size.

Step 14

Type the following command to attach the new disk to the jenkins-master instance.

gcloud compute instances \
attach-disk jenkins-master \
--disk data-02 \
--device-name disk02 \
--zone $CPO200_ZONE

The output from the command confirms that the instance has been updated:

Updated [.../jenkins-master].

Step 15

Return to the Cloud Platform Console and click the SSH button corresponding to the jenkins-master instance.

Step 16

Connect to the jenkins-master instance using the SSH command. Verify the attachment of the disk by typing the following command.

Instance: jenkins-master

ls -d /dev/disk/by-id/* | grep disk02

The output shows the the operating system has a reference to the newly attached disk.

/dev/disk/by-id/google-disk02

...

Step 17

Type the following command to mount the new disk to the mount point /mnt/disk02.

Instance: jenkins-master

sudo mount \
/dev/disk/by-id/google-disk02 \
/mnt/disk02

The command has no output.

Step 18

Confirm that the newly mounted, snapshot copy of the original disk contains the same data by typing the following command.

Instance: jenkins-master

sudo diff -rq /mnt/disk01 /mnt/disk02

The command has not output, indicating no differences between the two disks and verifying that a snapshot made of a mounted disk can be made cleanly.

Step 19

Type the following command to unmount the two disks.

Instance: jenkins-master

sudo umount /mnt/disk01 /mnt/disk02

The command has not output.

Step 20

Type the command to exit the connection to the jenkins-master instance.

exit

Step 21

Return to Cloud Shell and type the following two commands to detach the two disks from the jenkins-master instance:

gcloud compute instances \
detach-disk jenkins-master \
--disk data-01 \
--zone $CPO200_ZONE

gcloud compute instances \
detach-disk jenkins-master \
--disk data-02 \
--zone $CPO200_ZONE

Each command output confirms that jenkins-master has been updated.

Updated [.../jenkins-master].

Once you finish with the test instances, we recommend deleting them as you incur costs as long as they run.

Step 1

Type the Cloud SDK command to delete the jenkins-restore instance and associated boot persistent disk in your selected alternative zone.

gcloud compute instances \
delete jenkins-restore \
--zone $ALT_ZONE

The output asks you to confirm that you are sure you want to continue:

The following instances will be deleted. Attached disks configured to

be auto-deleted will be deleted unless they are attached to any other

instances. Deleting a disk is irreversible and any data on the disk

will be lost.

- [jenkins-restore] in [europe-west1-b]

Do you want to continue (Y/n)?

Type Y and press Enter to continue.

The output confirms that the resource has been deleted.

Step 2

Type the following command to delete the snapshot resource ‘jenkins-snapshot-1' you created in this lab.

gcloud compute snapshots delete \
jenkins-snapshot-1 data-01-snap

The output asks you to confirm that you are sure you want to continue:

The following snapshots will be deleted:

- [data-01-snap]

- [jenkins-snapshot-1]

Do you want to continue (Y/n)?

Type Y and press Enter to continue.

The output confirms that the resources have been deleted.

Step 3

Type the following command to delete the disks data-01 and data-02 you created while experimenting with snapshot creation:

gcloud compute disks \
delete data-01 data-02 \
--zone $CPO200_ZONE

The output asks you to confirm that you are sure you want to continue:

The following disks will be deleted. Deleting a disk is irreversible

and any data on the disk will be lost.

- [data-01] in [<your-default-zone>]

- [data-02] in [<your-default-zone>]

Do you want to continue (Y/n)?

Type Y and press Enter to continue.

The output confirms that the resources have been deleted.

Introduction

After completing this lab, your instructor will lead you through the following review activity. You can find a reference solution with suggestions at the end of the lab.

Q&A

Review the module slides, lab notes, and online documentation to answer the following questions.

What you learned

In this lab, you:

The answers to the lab challenge questions are as follows:

Step 1

In the next step, you select an alternative zone to restore the snapshot to. Type the Cloud SDK command to list the available Compute Engine zones and make a note of possible alternatives to your default zone. If you are not sure of the syntax, you can find the solution at the end of the lab.

gcloud compute zones list

Step 3

In this step, you observe how snapshots can be used to migrate an instance from one zone to another as well as change the type of disk. Can you identify any other potential uses for snapshots when managing the Jenkins master?

Snapshots can also be used to grow the size of the root file system for the Jenkins master or any attached volumes.

The answers to the lab review questions are as follows:

Snapshots are subject to quotas, for example 1000 snapshots per project by default. For example, you could not take 1 snapshot per day of 3 instances over a 1 year period.

Snapshots can't be shared across projects.

Disks must be quiesced prior to snapshot.

Snapshots cannot be used to perform disaster recovery tests in a separate project.

Snapshots are not a good fit for relational database backups due to the downtime they incur.

Snapshots are not a good fit for scenarios where disks cannot be quiesced.

©Google, Inc. or its affiliates. All rights reserved. Do not distribute.