Pic-a-daily: Lab 4—Create a web frontend

In this code lab, you create a web frontend on Google App Engine, that will let users upload pictures from the web application, as well as browse the uploaded pictures and their thumbnails.

1a5639c700d6ca09.png

This web application will be using a CSS framework called Bulma, for having some good looking user interface, and also the Vue.JS JavaScript frontend framework to call the application's API you will build.

This application will consist of three tabs:

  • A home page that will display the thumbnails of all the uploaded images, along with the list of labels describing the picture (the ones detected by the Cloud Vision API in a previous lab).
  • A collage page that will show the collage made of the 4 most recent pictures uploaded.
  • An upload page, where users can upload new pictures.

The resulting frontend looks as follows:

badd10dde09e663b.png

Those 3 pages are simple HTML pages:

  • The home page (index.html) calls the Node App Engine backend code to get the list of thumbnail pictures and their labels, via an AJAX call to the /api/pictures URL. The home page is using Vue.js for fetching this data.
  • The collage page (collage.html) points at the collage.png image that assembles the 4 latest pictures.
  • The upload page (upload.html) offers a simple form to upload a picture via a POST request to the /api/pictures URL.

What you'll learn

  • App Engine
  • Cloud Storage
  • Cloud Firestore

Self-paced environment setup

  1. Sign in to Cloud Console and create a new project or reuse an existing one. (If you don't already have a Gmail or G Suite account, you must create one.)

dMbN6g9RawQj_VXCSYpdYncY-DbaRzr2GbnwoV7jFf1u3avxJtmGPmKpMYgiaMH-qu80a_NJ9p2IIXFppYk8x3wyymZXavjglNLJJhuXieCem56H30hwXtd8PvXGpXJO9gEUDu3cZw

ci9Oe6PgnbNuSYlMyvbXF1JdQyiHoEgnhl4PlV_MFagm2ppzhueRkqX4eLjJllZco_2zCp0V0bpTupUSKji9KkQyWqj11pqit1K1faS1V6aFxLGQdkuzGp4rsQTan7F01iePL5DtqQ

8-tA_Lheyo8SscAVKrGii2coplQp2_D1Iosb2ViABY0UUO1A8cimXUu6Wf1R9zJIRExL5OB2j946aIiFtyKTzxDcNnuznmR45vZ2HMoK3o67jxuoUJCAnqvEX6NgPGFjCVNgASc-lg

Remember the project ID, a unique name across all Google Cloud projects (the name above has already been taken and will not work for you, sorry!). It will be referred to later in this codelab as PROJECT_ID.

  1. Next, you'll need to enable billing in Cloud Console in order to use Google Cloud resources.

Running through this codelab shouldn't cost much, if anything at all. Be sure to to follow any instructions in the "Cleaning up" section which advises you how to shut down resources so you don't incur billing beyond this tutorial. New users of Google Cloud are eligible for the $300USD Free Trial program.

Start Cloud Shell

While Google Cloud can be operated remotely from your laptop, in this codelab you will be using Google Cloud Shell, a command line environment running in the Cloud.

From the GCP Console click the Cloud Shell icon on the top right toolbar:

E0b6xMEnCN6XCtm5OITZ-CHPnhUsO3WrGGJFu0Yr587eWRPZG2xj4U9wHbTxF8d1LTHnk5yzgMxEbhAmTCwbNH8rMoQV70pEkLkz54gtUHD7kRtiSI_2EqrighTDFbuoO0Z146CC3Q

It should only take a few moments to provision and connect to the environment. When it is finished, you should see something like this:

kXnInzErAjsyeUStcIiTdNi179GwXpgp-2YTay2z0DW_7PoZ7uPWiKlaYk0LXNwv2kvkqUZEjhWjAgwNsgkX4Kpkhu8duXo5FTsog9bM405TSmdC_BUIX4ywkMV-tEc1VHtUzdTykg

This virtual machine is loaded with all the development tools you'll need. It offers a persistent 5GB home directory, and runs on Google Cloud, greatly enhancing network performance and authentication. All of your work in this lab can be done with simply a browser.

Checkout the code, if you haven't already:

git clone https://github.com/GoogleCloudPlatform/serverless-photosharing-workshop

You can then go to the directory containing the frontend:

cd serverless-photosharing-workshop/frontend

You will have the following file layout for the frontend:

frontend
 |
 ├── index.js
 ├── package.json
 ├── app.yaml
 |
 ├── public
      |
      ├── index.html
      ├── collage.html
      ├── upload.html
      |
      ├── app.js
      ├── style.css

At the root of our project, you have 3 files:

  • index.js contains the Node.js code
  • package.json defines the library dependencies
  • app.yaml is the configuration file for Google App Engine

A public folder contains static resources:

  • index.html is the page showing all the thumbnail pictures and labels,
  • collage.html shows the collage of the recent pictures,
  • upload.html contains a form to upload new pictures.
  • app.js is using Vue.js to populate the index.html page with the data
  • style.css defines some CSS directives

Dependencies

The package.json file defines the needed library dependencies:

{
  "name": "frontend",
  "version": "0.0.1",
  "main": "index.js",
  "scripts": {
    "start": "node index.js"
  },
  "dependencies": {
    "@google-cloud/firestore": "^3.4.1",
    "@google-cloud/storage": "^4.0.0",
    "express": "^4.16.4",
    "dayjs": "^1.8.22",
    "bluebird": "^3.5.0",
    "express-fileupload": "^1.1.6"
  }
}

Our application depends on:

  • firestore: to access Cloud Firestore with our picture metadata,
  • storage: to access Google Cloud Storage where pictures are stored,
  • express: the web framework for Node.js,
  • dayjs: a small library to show dates in a human-friendly way,
  • bluebird: a JavaScript promise library,
  • express-fileupload: a library to handle file uploads easily.

Express frontend

At the beginning of the index.js controller, you will require all the dependencies defined in package.json earlier:

const express = require('express');
const fileUpload = require('express-fileupload');
const Firestore = require('@google-cloud/firestore');
const Promise = require("bluebird");
const {Storage} = require('@google-cloud/storage');
const storage = new Storage();
const path = require('path');
const dayjs = require('dayjs');
const relativeTime = require('dayjs/plugin/relativeTime')
dayjs.extend(relativeTime)

Next, the Express application instance is created.

Two Express middleware are used:

  • The express.static() call indicates that static resources will be available in the public sub-directory.
  • And fileUpload() configures file upload to limit file size to 10 MB, to upload the files locally in the in-memory file system in the /tmp directory.
const app = express();
app.use(express.static('public'));
app.use(fileUpload({
    limits: { fileSize: 10 * 1024 * 1024 },
    useTempFiles : true,
    tempFileDir : '/tmp/'
}))

Among the static resources, you have the HTML files for the home page, the collage page, and the upload page. Those pages will call the API backend. This API will have the following endpoints:

  • POST /api/pictures Via the form in upload.html, pictures will be uploaded via a POST request
  • GET /api/pictures This endpoint returns a JSON document containing the list of pictures and their labels
  • GET /api/pictures/:name This URL redirects to the cloud storage location of the full-size image
  • GET /api/thumbnails/:name This URL redirects to the cloud storage location of the thumbnail image
  • GET /api/collage This last URL redirects to the the cloud storage location of the generated collage image

Picture upload

Before exploring the picture upload Node.js code, take a quick look at public/upload.html.

... 
<form method="POST" action="/api/pictures" enctype="multipart/form-data">
    ... 
    <input type="file" name="picture">
    <button>Submit</button>
    ... 
</form>
... 

The form element points at the /api/pictures endpoint, with an HTTP POST method, and a multi-part format. The index.js now has to respond to that endpoint and method, and extract the file content:

app.post('/api/pictures', async (req, res) => {
    if (!req.files || Object.keys(req.files).length === 0) {
        console.log("No file uploaded");
        return res.status(400).send('No file was uploaded.');
    }
    console.log(`Receiving file ${JSON.stringify(req.files.picture)}`);

    const newPicture = path.resolve('/tmp', req.files.picture.name);
    const mv = Promise.promisify(req.files.picture.mv);
    await mv(newPicture);
    console.log('File moved in temporary directory');

    const pictureBucket = storage.bucket(process.env.BUCKET_PICTURES);
    await pictureBucket.upload(newPicture, { resumable: false });
    console.log("Uploaded new picture into Cloud Storage");

    res.redirect('/');
});

First, you check that there is indeed a file being uploaded. Then you download the file locally via the mv method coming from our file upload Node module. Now that the file is available on the local filesystem, we upload the picture to the Cloud Storage bucket. Finally, we return back the user to the main screen of the application.

Listing the pictures

Time to display your beautiful pictures!

In the /api/pictures handler, you look into the Firestore database's pictures collection, to retrieve all the pictures (whose thumbnail has been generated), ordered by descending date of creation.

You push each picture in a JavaScript array, with its name, the labels describing it (coming from the Cloud Vision API), the dominant color, and a friendly date of creation (with dayjs, we relative time offsets like "3 days from now").

app.get('/api/pictures', async (req, res) => {
    console.log('Retrieving list of pictures');

    const thumbnails = [];
    const pictureStore = new Firestore().collection('pictures');
    const snapshot = await pictureStore
        .where('thumbnail', '==', true)
        .orderBy('created', 'desc').get();

    if (snapshot.empty) {
        console.log('No pictures found');
    } else {
        snapshot.forEach(doc => {
            const pic = doc.data();
            thumbnails.push({
                name: doc.id,
                labels: pic.labels,
                color: pic.color,
                created: dayjs(pic.created.toDate()).fromNow()
            });
        });
    }
    console.table(thumbnails);
    res.send(thumbnails);
});

This controller returns results of the following shape:

[
   {
      "name": "IMG_20180423_163745.jpg",
      "labels": [
         "Dish",
         "Food",
         "Cuisine",
         "Ingredient",
         "Orange chicken",
         "Produce",
         "Meat",
         "Staple food"
      ],
      "color": "#e78012",
      "created": "a day ago"
   },
   ...
]

This data structure is consumed by a small Vue.js snippet from the index.html page. Here's a simplified version of the markup from that page:

<div id="app">
   <div v-for="n in Math.ceil(pictures.length / 4)">
      <div v-for="pic in pictures.slice((n-1) * 4, n * 4)">
         <div :style="{ 'border-color': pic.color }">
            <a :href="'/api/pictures/' + pic.name">
               <img :src="'/api/thumbnails/' + pic.name">
                  </a>
         </div>
         <a v-for="label in pic.labels">
            {{ label }}
         </a>
      </div>
   </div>
</div>

The div's ID will indicate Vue.js that it's the part of the markup that will be dynamically rendered. This code is creating a 4-column view of the pictures, hence the bit of maths and array slicing to get 4 pictures per row. The iterations are done thanks to the v-for directives.

The pictures get a nice colored border corresponding to the dominant color in the picture, as found by the Cloud Vision API and we point at the thumbnails and the full-width pictures in the link and image sources.

Last, we list the labels describing the picture.

Here is the JavaScript code for the Vue.js snippet (in the public/app.js file imported at the bottom of the index.html page):

var app = new Vue({
  el: '#app',
  data() {
    return { pictures: [] }
  },
  mounted() {
    axios
      .get('/api/pictures')
      .then(response => { this.pictures = response.data })
  }
})

The Vue code is using the Axios library to make an AJAX call to our /api/pictures endpoint. The returned data is then bound to the view code in the markup you saw earlier.

Viewing the pictures

From the index.html our users can view the thumbnails of the pictures, click on them to view the full-size images, and fom collage.html, users view the collage.png image.

In the HTML markup of those pages, the image src and link href point at those 3 endpoints, which redirect to the Cloud Storage locations of the pictures, thumbnails, and collage. No need to hard-code the path in the HTML markup.

app.get('/api/pictures/:name', async (req, res) => {
    res.redirect(`https://storage.cloud.google.com/${process.env.BUCKET_PICTURES}/${req.params.name}`);
});

app.get('/api/thumbnails/:name', async (req, res) => {
    res.redirect(`https://storage.cloud.google.com/${process.env.BUCKET_THUMBNAILS}/${req.params.name}`);
});

app.get('/api/collage', async (req, res) => {
    res.redirect(`https://storage.cloud.google.com/${process.env.BUCKET_THUMBNAILS}/collage.png`);
});

Running the Node application

With all the endpoints defined, your Node.js application is ready to be launched. The Express application listens on port 8080 by default, and is ready to serve incoming requests.

const PORT = process.env.PORT || 8080;

app.listen(PORT, () => {
    console.log(`Started web frontend service on port ${PORT}`);
    console.log(`- Pictures bucket = ${process.env.BUCKET_PICTURES}`);
    console.log(`- Thumbnails bucket = ${process.env.BUCKET_THUMBNAILS}`);
});

Test the code locally to make sure it works before deploying to cloud.

You need to export the two environment variables corresponding to the two Cloud Storage buckets:

export BUCKET_THUMBNAILS=thumbnails-${GOOGLE_CLOUD_PROJECT}
export BUCKET_PICTURES=uploaded-pictures-${GOOGLE_CLOUD_PROJECT}

Inside frontend folder, install npm dependencies and start the server:

npm install; npm start

If everything went well, it should start the server on port 8080:

Started web frontend service on port 8080
- Pictures bucket = uploaded-pictures-${GOOGLE_CLOUD_PROJECT}
- Thumbnails bucket = thumbnails-${GOOGLE_CLOUD_PROJECT}

The real names of your buckets will appear in those logs, which is helpful for debugging purposes.

From Cloud Shell, you can use the web preview feature, to browser the application running locally:

b6c2f9801006a34b.png

Use CTRL-C to exit.

Your application is ready to be deployed.

Configure App Engine

Examine the app.yaml configuration file for App Engine:

runtime: nodejs10
env_variables:
  BUCKET_PICTURES: uploaded-pictures-GOOGLE_CLOUD_PROJECT
  BUCKET_THUMBNAILS: thumbnails-GOOGLE_CLOUD_PROJECT

The first line declares that the runtime is based on Node.js 10. Two environment variables are defined to point at the two buckets, for the original images and for the thumbnails.

To replace GOOGLE_CLOUD_PROJECT with your actual project id, you can run the following command:

sed -i -e "s/GOOGLE_CLOUD_PROJECT/${GOOGLE_CLOUD_PROJECT}/" app.yaml

Deploy

Set your preferred zone for App Engine, be sure to use the same region in the previous labs:

gcloud config set compute/zone europe-west1

And deploy:

gcloud app deploy

After a minute or two, you will be told that the application is serving traffic:

Beginning deployment of service [default]...
╔════════════════════════════════════════════════════════════╗
╠═ Uploading 8 files to Google Cloud Storage                ═╣
╚════════════════════════════════════════════════════════════╝
File upload done.
Updating service [default]...done.
Setting traffic split for service [default]...done.
Deployed service [default] to [https://GOOGLE_CLOUD_PROJECT.appspot.com]
You can stream logs from the command line by running:
  $ gcloud app logs tail -s default
To view your application in the web browser run:
  $ gcloud app browse

You can also visit the App Engine section of Cloud Console to see that the app is deployed and explore features of App Engine like versioning and traffic splitting:

f4bd5f4de028bd83.png

To test, go to the default App Engine url for the app (https://&lt;YOUR_PROJECT_ID&gt;.appspot.com/) app and you should see the frontend UI up and running!

badd10dde09e663b.png

If you don't intend to keep the app, you can clean up resources to save costs and to be an overall good cloud citizen by deleting the whole project:

gcloud projects delete ${GOOGLE_CLOUD_PROJECT} 

Congratulations! This Node.js web application hosted on App Engine binds all your services together, and allows your users to upload and visualize pictures.

What we've covered

  • App Engine
  • Cloud Storage
  • Cloud Firestore