In this lab, you continue to explore Google App Engine using the Bookshelf sample application. You deploy a new version of Bookshelf that uses Google Cloud Storage to store book image covers. You deploy the application to App Engine, and then you test it in your browser. You then review an uploaded object in Cloud Storage using the Cloud Platform Console.
To complete this lab, you need:
Access to a supported Internet browser:
A Google Cloud Platform project
A Source Code repository containing the lab code
In this lab, you:
In this lab, you integrate Google Cloud Storage into the Google App Engine Bookshelf sample application. You continue to use Google Cloud Datastore to store structured data for books. You use Google Cloud Storage to store book cover images, using both storage services side-by-side.
Google Cloud Storage offers developers and IT organizations durable and highly-available object storage. Applications typically require access to a variety of storage systems for different types of data. As you observed earlier in the course, storage services such as Cloud Datastore are designed to handle more structured data requirements. Other types of unstructured data, including images, video, and audio with much larger file sizes, are typically stored as Binary Large Objects (BLOB), and Cloud Storage is an ideal fit for this type of requirement.
Google Cloud Storage includes support for a JSON RESTful API, in addition to a variety of other access methods. Developers building applications for the standard App Engine Python environment can make use of a dedicated Cloud Storage client library, that greatly simplifies accessing the service. The version of Bookshelf used in this lab makes use of this client library and you review a code sample before deploying the application to App Engine. Later in the course, you deploy alternative versions of Bookshelf to Google Container Engine and Google Compute Engine, that make use of the Google Cloud Client Library for Python.
The following diagram illustrates the high-level components and resources that make up the version of Bookshelf you deploy in this lab.
When you store data in Google Cloud Storage, you choose a storage class and location to find the right balance of availability, latency, and price for your data. Once you choose a class and location, you organize your data in containers called buckets. Buckets store the data (blobs) you upload.
The structure of data in Cloud Storage looks like the following.
You can create and manage multiple buckets within a project, and each bucket you create shares a global namespace (it must be globally unique). When you create a bucket, it cannot be nested inside another bucket.
The objects you upload to Cloud Storage are comprised of data and metadata. Metadata is used to control how the data is accessed (via GET requests). When you upload objects to Cloud Storage, they cannot be nested.
Before deploying your application, you create a Google Cloud Storage bucket to store the book cover images. You also review the code that accesses Cloud Storage from your App Engine application.
To create a bucket:
Open the Google Cloud Platform Console, and if necessary, select the cp100 project.
In the top right corner of the console window, click the Activate Google Cloud Shell button ().
To make use of Google Cloud Storage from App Engine, you first need to create a bucket that is used to host objects containing the book cover images. It is possible to create buckets using the relevant APIs, but as this is a one time operation, you use the command line to create the resource. You could also carry out the same operation using the Cloud Platform Console.
In this step you use the
gsutil command to create a bucket. This command is included in Cloud Shell and can also be installed as part of the Cloud SDK. The
mb command is used to make buckets. Notice that bucket names begin with the
gs:// URI prefix.
Type the following command in Cloud Shell to create a Cloud Storage bucket with the same name as your project ID. Specify the location for your bucket using the
-l flag as indicated below (the flag is a lowercase L, not a number one). Available location options are ASIA, EU, or US. You should create the bucket in the same region as your App Engine application. You can find the location of your application in the top right-hand corner of the App Engine Dashboard, by visiting Products & services > App Engine in the Cloud Platform Console.
gsutil mb -l <location> gs://$DEVSHELL_PROJECT_ID
For example, if you are running App Engine in Europe, then the command would read as follows.
gsutil mb -l EU gs://$DEVSHELL_PROJECT_ID
The output confirms that the bucket is being created.
The version of Bookshelf used in this course does not include any authorization or authentication controls for the book data added by users. For the sake of this simple illustration, any images uploaded when adding details for a book should be publically visible by anonymous users.
In this step you use the
defacl command to give
AllUsers read access to your bucket. Type the following command to configure the default ACL for your new bucket to ensure all new objects are publically readable.
gsutil defacl ch -u AllUsers:R gs://$DEVSHELL_PROJECT_ID
The output confirms that the default ACL is updated.
You also need to configure the source code for Bookshelf with the name of your new bucket before you can deploy it to App Engine.
Return to the Cloud Platform Console and click Tools > Source Repositories > Source Code.
Click cloud-storage > config.py.
This file is used to set configuration data specific to your copy of Bookshelf in this project. Settings related to Cloud Storage are located on lines 47 through 49. The settings include the name of the bucket you created for this lab, as well as the list of supported file extensions for Bookshelf.
Later in the lab, you use a command line tool to modify this file with the value of your bucket name.
Before you deploy your code, it is useful to also inspect the new code that relates to the new book cover features you test in this lab.
The code for this version of Bookshelf remains largely unchanged from the one you previously uploaded to App Engine. Most of the code relating to integrating Cloud Storage with App Engine can be found in a single file that you inspect next.
Click cloud-storage > bookshelf > storage.py.
This file includes a number of methods that relate to uploading images to Bookshelf, including checking against a whitelist of allowed filename extensions for popular image formats. It also generates a unique time-stamped name for the new object as you observe later in the lab.
Note: You can find a list of the allowed file extensions in
upload_file() function, on lines 47 through 69, coordinates the upload operation, including:
There is one last file you should investigate.
Click cloud-storage > bookshelf > templates > list.html.
The HTML template files provide the browser with the direct links to images hosted on Cloud Storage. As you observed in the previous step, once an image has been uploaded, the Object URI is returned by
upload_file(). On lines 31 through 35, this template file checks if a Book entity has a valid
imageUrl property and retrieves the image from your bucket. If no
imageUrl is found then a default value is used on line 34.
Notice that this default URI points to the same Cloud Storage bucket used to download the sample images. This URI format is the same as the URI stored in Cloud Datastore (once you upload an image when creating a new book entry). You might like to review the HTML source code in your browser to see how the URIs appear (once you add additional books with sample covers).
Leave the Cloud Platform Console and Cloud Shell windows open.
Install all required third-party libraries and dependencies to the local clone of your repository in Cloud Shell. Deploy the application to App Engine using the Cloud SDK.
To deploy the application:
Return to Cloud Shell and type the following command to change to the directory containing the code for this lab.
Type the following command to replace the bucket name placeholder in
config.py with the value of your project ID. The text ‘your-bucket-name' in the command refers to the text the command will replace.
sed -i s/your-bucket-name/$DEVSHELL_PROJECT_ID/ config.py
There is no output for this command.
As with the previous deployment of Bookshelf to App Engine, you also need to install a number of libraries that the Bookshelf application uses.
If you would like to see a list of the libraries you need for this lab, type the following command to view the contents of the
requirements.txt text file.
The file lists package names and version numbers that you install in the next step.
Note in particular that this version of the application makes use of the
GoogleAppEngineCloudStorageClient package. This client library simplifies integrating App Engine applications with Cloud Storage as you observed earlier in the lab when examining the code for
Type the following command to install the packages and dependencies required for this lab into the
pip install -r requirements.txt -t lib
The output displays the progress of the installations.
You are ready to deploy the code for Bookshelf to App Engine. Once again, you use the
gcloud utility to manage your App Engine application. Type the following command to deploy the application.
gcloud app deploy
You are asked to confirm if you want to proceed with the deployment:
Do you want to continue (Y/n)?
Type Y and press return.
The output displays the progress of the deployment.
Type the following command to quit Cloud Shell.
Leave the Cloud Platform Console window open.
Generate some simple test data in the Bookshelf application, including a sample cover image, to verify that it is working as expected. View the resulting Cloud Storage test object using the Cloud Platform Console.
To test Bookshelf:
Open a new tab in your browser and visit your Bookshelf App Engine application.
As this version of Bookshelf includes support for uploading book covers, you need some sample images to use when testing the application. You can download a ZIP archive containing all of the sample images for this course using the following button. You will need to unzip the archive to upload the image required in this lab.
Alternatively, you can find a link to the specific image you need for this lab in the relevant step below.
You may, or may not see existing book details in the bookshelf, based on any previous labs you may have completed earlier in the course.
Click Add Book.
In the ‘Add book' page:
Note: The link above can be used to retrieve the specific image for this step.
A page loads with the details you recorded for the new book, in addition to the sample cover image.
Return to the Cloud Platform Console browser tab and click Storage > Storage.
The Cloud Storage Browser loads.
Click the name of the bucket you created to host book cover images. Recall from earlier in the lab that this name is the same value as your project ID.
You are presented with a list of objects in this particular bucket. You should currently have a single object in the bucket with the following name format.
You can also see other details in this view relating to objects, including their size, type, and modified times.
Optionally, if you want to view the image in your browser, you can click the name of the object and it will load in a new tab.
Close all browser tabs.
There is no need for you to remove the resources used in this lab.
©Google, Inc. or its affiliates. All rights reserved. Do not distribute.