In this codelab, you'll learn how to integrate Google Cloud Platform services into a Node.js web application to store data, upload images, and authenticate users.

What you'll learn

What you'll need

How will you use use this tutorial?

Read it through only Read it and complete the exercises

How would you rate your experience with building node.js apps?

Novice Intermediate Proficient

How would you rate your experience with using Google Cloud Platform services?

Novice Intermediate Proficient

Self-paced environment setup

If you don't already have a Google Account (Gmail or Google Apps), you must create one. Sign-in to Google Cloud Platform console (console.cloud.google.com) and create a new project:

Remember the project ID, a unique name across all Google Cloud projects (the name above has already been taken and will not work for you, sorry!). It will be referred to later in this codelab as PROJECT_ID.

Next, you'll need to enable billing in the Developers Console in order to use Google Cloud resources like Cloud Datastore and Cloud Storage.

Running through this codelab shouldn't cost you more than a few dollars, but it could be more if you decide to use more resources or if you leave them running (see "cleanup" section at the end of this document). Prices are documented here: Cloud Datastore & Cloud Storage

New users of Google Cloud Platform are eligible for a $300 free trial.

While 'Build a Node.js Web App using Google Cloud Platform' can be operated remotely from your laptop, in this codelab we will be using Google Cloud Shell, a command line environment running in the Cloud.

This Debian-based virtual machine is loaded with all the development tools you'll need. It offers a persistent 5GB home directory, and runs on the Google Cloud, greatly enhancing network performance and authentication. This means that all you will need for this codelab is a browser (yes, it works on a Chromebook).

To activate Google Cloud Shell, from the developer console simply click the button on the top right-hand side (it should only take a few moments to provision and connect to the environment):

Once connected to the cloud shell, you should see that you are already authenticated and that the project is already set to your PROJECT_ID :

gcloud auth list

Command output

Credentialed accounts:
 - <myaccount>@<mydomain>.com (active)
gcloud config list project

Command output

[core]
project = <PROJECT_ID>

If for some reason the project is not set, simply issue the following command :

gcloud config set project <PROJECT_ID>

Looking for your PROJECT_ID? Check out what ID you used in the setup steps or look it up in the console dashboard:

IMPORTANT: Finally, set the default zone and project configuration:

gcloud config set compute/zone us-central1-f

You can choose a variety of different zones. Learn more in the Regions & Zones documentation.

In Cloud Shell on the command-line, run the following command to clone the Github repository:

git clone https://github.com/googlecodelabs/cloud-nodejs.git

Change directory into cloud-nodejs/start.

cd cloud-nodejs/start

The sample has the following layout:

app.js        /* Express.js web application */
books.js      /* Code for creating, deleting, and listing books */
auth.js       /* Code for user authentication */
config.js     /* Application configuration variables */
package.json  /* npm package file including dependencies */
views/
  index.jade  /* HTML template */
public/
  style.css   /* CSS stylesheet */

To run the sample application on your local computer, let's perform the following steps:

1. Install dependencies. Enter the following command:

npm install

2. Run app.js to start the node.js server:

node app.js

3. Click the "Web preview"icon that appears at the top left side of the cloud shell window and select "Preview on port 8080" to see the app in a web browser.

You will see a page that looks like this:

The application currently displays a fake book.

Let's fix that by querying for books from datastore!

Summary

In this step, you set up and ran the codelab sample application.

Next up

Next, you will setup Google Cloud Datastore and configure the application to begin querying it.

In this step, you will set up Google Cloud Datastore and configure the application to query from it.

Create Credentials

For the node.js application to access this project's services, eg. Datastore and Cloud Storage, it needs to be authenticated. Create a Service Account for this project which will be used to authenticate the application.

  1. In the Cloud Console, navigate to API Manager > Credentials
  2. Make sure your project is selected and click Continue
  3. Next, click Create credentials
  4. Select "Service account key" from the drop-down menu
  5. Under the Service Account dropdown, select "Compute Engine default service account" and ensure that Key type is set to JSON. Then click the "Create" button
  6. Once the JSON key file automatically downloads, locate it (it should have a name of the form) <project name>-<hash string>.json and open it in a text editor.
  7. In the "cloud-nodejs/start" directory, copy and paste the contents of the file into a new file called key.json in the Cloud Shell window.

Enable Datastore API

The credentials you created allow your application to communicate with Google APIs that you enable for this project. Enable the Datastore API so the application can access Datastore.

  1. Under API Manager, click Enable API and search for "Cloud Datastore API"
  2. Click on Cloud Datastore API
  3. Click Enable (at the top of the page).

Update configuration variables

To configure the node.js sample application to authenticate with your project, go back to your Cloud Shell instance and edit the config.js file in the "cloud-nodejs/start" directory and replace the placeholder value for projectId with the Project ID of your project:

config.js

module.exports = {
  projectId: '[your Google Developers Console project ID]',
  keyFilename: './key.json',
  // ...
};

Summary

In this step, you created a Google Developer Console project and configured the node.js application with the credentials needed to query Datastore.

Next up

Next, you will update the application to query books from your project's Datastore.

The sample application's home page lists all books.

Books are retrieved by calling books.getAllBooks from the books module.

app.js

var config = require('./config');
var books = require('./books')(config);
// ...

/* Fetch all books and display them */
app.get('/', function(req, res, next) {
  books.getAllBooks(function(err, books) {
    if (err) return next(err);
    res.render('index', { books: books, user: req.session.user });
  });                                                                              
});

The current implementation of getAllBooks found in books.js simply returns a fake book.

books.js

function getAllBooks(callback) {                                                
  var error = null;
  var books = [
    {
      key: { path: ['Book', 12345] },
      data: { title: 'Fake Book', author: 'Fake Author' }
    }
  ];
  callback(error, books);
}

In this step, you will write the code for getAllBooks to query for Book entities from Datastore.

Configure Datastore

To begin, install the gcloud npm package, which you will use to interact with Cloud Datastore.

npm install gcloud --save

In the project directory, edit the books.js file.

Copy the following code block:

  var gcloud = require('gcloud');

  var datastore = gcloud.datastore({
    projectId: config.projectId,
    keyFilename: config.keyFilename
  });

Then add the copied code block to the books.js file:

books.js

module.exports = function(config) {

   // Add the copied code block here

  function getAllBooks(callback) {
    // ...

The gcloud.datastore object allows you to interact with Google Cloud Datastore

The projectId and keyFilename are read from the config.js file that you edited earlier.

Query for entities

Now replace the current getAllBooks function with the following:

books.js

function getAllBooks(callback) {
  var query = datastore.createQuery(['Book']);
  datastore.runQuery(query, callback);                                                   
}

Datastore queries are build with datastore.createQuery and run via datastore.runQuery.

createQuery accepts an array containing the kinds of entities to query and returns a Query object.

In our case, we want to query for all 'Book' entities.

runQuery runs a query and returns its results as a list of entities in the following format:

{
  key: { path: ['Book', <Numeric entity ID>] },
  data: {
    title: 'A Tale of Two Cities',
    author: 'Charles Dickens',
    imageUrl: 'http://books.google.com/books...'
  }
}

To view your changes, stop your running node.js web server by pressing CTRL + C and run it again.

node app.js

Click the "Web preview" icon that appears at the top left side of the cloud shell window and select "Preview on port 8080" to see the app in a web browser.

Now, you should see no books because there are none in your Datastore!

Create Entity using the Developers Console

To test the Datastore query and see data displayed, add a Book entity from the Google Cloud Console.

  1. Open a new web browser taband visit Google Cloud Console
  2. Select to your Google Cloud Platform project using the dropdown at the top.
  3. From the left navigation menu under "Storage", click Datastore
  4. Click Create Entity

Fill out the Create an entity form with the following:

  1. Namespace: Leave this set as [default]
  2. Kind: Select Book from the dropdown
  3. Key Identifier Type: leave the default value of "Numeric ID (auto-generated)"
  4. Under the property list, change the value of the title property to A Tale of Two Cities
  5. Then, change the value of the author property to Charles Dickens
  6. Click Save

Now refresh the app in your browser and you should see the book listed!

Summary

In this step, you queried for all Book entities from Google Cloud Datastore.

Next up

Next, you will write the code to create and delete books.

Create books

The sample application includes a form for adding books. When the form is submitted, the application creates the book by calling books.addBook.

app.js

/* Add a new book */
app.post('/books', function(req, res, next) {
  // ...

  books.addBook(req.body.title, req.body.author, coverImageData, userId, function(err) {
    if (err) return next(err);
    res.redirect(req.get('Referer') || '/');
  })  
});

The book title and author come from the 'title' and 'body' fields in the form.

You can ignore coverImageData and userId for now - we'll come back to these later :)

Currently, if you try to add a book, you receive an error:

Let's fix that!

Creating books

Press CTRL + C in Cloud Shell to exit the running Node.js app. Then, in the project directory, edit the books.js file.

The current addBook function is a placeholder that simply returns an error.

books.js

function addBook(title, author, coverImageData, userId, callback) {
  if (coverImageData)
    return callback(new Error("books.addBook image saving Not Yet Implemented"));

  return callback(new Error("books.addBook datastore saving Not Yet Implemented"));
}

To save a new Book entity in Datastore, replace the current addBook function with the following.

books.js

function addBook(title, author, coverImageData, userId, callback) {
  if (coverImageData)
    return callback(new Error("books.addBook image saving Not Yet Implemented"));

  var entity = {
    key: datastore.key('Book'),
    data: {
      title: title,
      author: author
    }
  };

  datastore.save(entity, callback);                                               
}

datastore.save inserts or updates entity objects.

Entity objects are represented the following format:

{
  key: datastore.key('Book', [Numeric entity ID]) },
  data: {
    title: 'A Tale of Two Cities',
    author: 'Charles Dickens'
    // ... properties ...
  }
}

If an entity ID is provided, calling save will update the entity in Datastore. In our example, the book entity is being added to Datastore for the first time, so no ID is necessary (an ID will be auto-generated by Datastore).

Restart your webserver (node app.js) and try submitting the form again with different values to add a new book.

You should now see your added book!

Deleting books

Now try deleting a book by clicking the delete link below it. You will receive an error:

Press CTRL-C to stop the webserver, and take a look at books.js. The delete link in the sample application deletes books by calling books.deleteBook with the ID of the book to delete.

app.js

/* Delete book by key */
app.get('/books/delete', function(req, res, next) {
  books.deleteBook(req.query.id, function(err) {
    if (err) return next(err);
    res.redirect('/');
  }); 
});

The sample implementation of books.deleteBook is also a placeholder which simply returns an error:

books.js

function deleteBook(bookId, callback) {
  callback(new Error("books.deleteBook datastore deletion Not Yet Implemented"));
}

To delete the book with the provided ID, replace the current deleteBook function with the following.

books.js

function deleteBook(bookId, callback) {
  var key = datastore.key(['Book', parseInt(bookId, 10)]);
  datastore.delete(key, callback);                                          
}

Restart the node application and try deleting book again.

It should work!

Summary

In this step, you wrote the code to create and delete entities in Datastore.

Next up

Next, you will use Google Cloud Storage to upload book cover images.

The form for adding books also allows you to attach a cover image for a book.

If you try to create a new book with an attached image, you receive an error:

In this step, you will write the code to save book cover images in Google Cloud Storage.

Set up Google Cloud Storage

Press CTRL-C to stop the webserver.

Then, create a Cloud Storage bucket to store book cover images:

  1. Open a new web browser tab and visit Google Cloud Console
  2. Select to your Google Cloud Platform project from the dropdown menu
  3. From the left navigation menu , find the Storage section and click Storage
  4. Click Create bucket
  5. Choose a unique bucket name and click Create
  6. Note the bucket name since you'll need it for configuration

Go back to the tab with Cloud Shell in it. In the project directory, edit the config.js file and replace the placeholder value for bucketName with the name of the bucket you created:

config.js

module.exports = {
  // ...
  bucketName: '[your Google Cloud Storage bucket name]',
  // ...
};

Upload Images to Cloud Storage

When the form is submitted with an image attached, the image data from the cover form field is passed to the books.addBook function:

app.js

app.use(multer({ inMemory: true })); // store upload data in-memory
// ...

/* Add a new book */
app.post('/books', function(req, res, next) {
  // ...

  var coverImageData;
  if (req.files['cover'])
    coverImageData = req.files['cover'].buffer;

  // ...

  books.addBook(req.body.title, req.body.author, coverImageData, userId, function(err) {
    if (err) return next(err);
    res.redirect(req.get('Referer') || '/');
  })                                                                            
});

Currently, addBook returns an error when image data is provided:

books.js

function addBook(title, author, coverImageData, userId, callback) {                        
  if (coverImageData)                                                              
    return callback(new Error("books.addBook image saving Not Yet Implemented"));
                                                                                
  // ...                                           
}

Let's fix this!

When coverImageData is passed in, we will save the book in Datastore with an imageUrl property specifying the URL to the uploaded book cover image.

To handle this scenario, replace the current addBook function in books.js with the following.

books.js

function addBook(title, author, coverImageData, userId, callback) {                        
  var entity = {                                                                   
    key: datastore.key('Book'),                                                      
    data: {                                                                        
      title: title,                                                                
      author: author                                                               
    }                                                                              
  };                                                                               
                                                                                
  if (coverImageData)                                                            
    uploadCoverImage(coverImageData, function(err, imageUrl) {                  
      if (err) return callback(err);                                            
      entity.data.imageUrl = imageUrl;                                               
      datastore.save(entity, callback);                                           
    });                                                                         
  else                                                                          
    datastore.save(entity, callback);                                                
}

We have not yet implemented uploadCoverImage so trying to add a book with an image still results in an error:

Let's fix this now!

Uploading to Google Cloud Storage

Edit the books.js file and add storage and bucket variables in the location shown below:

books.js

Copy the following code block:

  var storage = gcloud.storage({
    projectId: config.projectId,
    keyFilename: config.keyFilename
  });

  var bucket = storage.bucket(config.bucketName);

Then, add it to books.js as shown below:

module.exports = function(config) {

  var gcloud = require('gcloud')({                                                                                                                                                                 
    projectId: config.projectId,                                                  
    keyFilename: config.keyFilename                                               
  });

  var datastore = gcloud.datastore({
    projectId: config.projectId,
    keyFilename: config.keyFilename
  });

  // Add your copied code here
  
  // ...

bucketName is read from the config.js file that you edited earlier.

The bucket object provides the API you will use to interact with your Google Cloud Storage bucket.

To upload the image to Cloud Storage and return a publicly accessible URL for displaying the image, add the following uploadCoverImage function to books.js:

books.js

function uploadCoverImage(coverImageData, callback) {
  // Generate a unique filename for this image
  var filename = '' + new Date().getTime() + "-" + Math.random();
  var file = bucket.file(filename);
  var imageUrl = 'https://' + config.bucketName + '.storage.googleapis.com/' + filename;
  var stream = file.createWriteStream();
  stream.on('error', callback);
  stream.on('finish', function() {
    // Set this file to be publicly readable
    file.makePublic(function(err) {
      if (err) return callback(err);
      callback(null, imageUrl);
    });
  });
  stream.end(coverImageData);
}

bucket.file returns a File object providing the API for a file in Cloud Storage.

The publicly accessible URL for the image file will be https://<bucket name>.storage.googleapis.com/<filename>

Restart the node application and try adding a book with an image again.

It should work!

Deleting images

Right now, if you delete a book that has a cover image, the image will remain in your Cloud Storage bucket.

Let's fix this by updating deleteBook to also delete saved cover images.

Replace the current deleteBook function with the following:

books.js

function deleteBook(bookId, callback) {
  var key = datastore.key(['Book', parseInt(bookId, 10)]);

  datastore.get(key, function(err, book) {
    if (err) return callback(err);

    if (book.data.imageUrl) {
      var filename = url.parse(book.data.imageUrl).path.replace('/', '');
      var file = bucket.file(filename);
      file.delete(function(err) {
        if (err) return callback(err);
        datastore.delete(key, callback);
      });
    } else {
      datastore.delete(key, callback);
    }
  });
}

Before deleting a book, the book is retrieved from Datastore by ID via datastore.get to determine whether or not it has a cover image.

If the book has a cover image, the file is deleted from Cloud Storage via file.delete before deleting the entity from Datastore.

Now deleting books with cover images will also delete the images from your bucket!

Summary

In this step, you created a Google Cloud Storage bucket, uploaded images into it, and deleted images from it.

Next up

Next, you will use OAuth 2.0 to add user login to the application.

The sample application includes a ‘Sign in' link so that users can login and see a list of only the books they have added.

Currently, clicking ‘Sign in' logs you in as "Fake User."

Let's fix this!

When a user logs in, the sample application should redirect the user to an authentication URL - in this case, the Google login screen.

Next, you will associate users with books

Once the user is authenticated, they will be redirected back to your application with an authorization code you can use to get their profile information.

The application gets the authentication URL to redirect to by calling auth.getAuthenticationUrl from auth.js.

app.js

/* Redirect user to OAuth 2.0 login URL */
app.get('/login', function(req, res) {
  var authenticationUrl = auth.getAuthenticationUrl();
  res.redirect(authenticationUrl);
});

Right now, getAuthenticationUrl is a placeholder that simply redirects to the callback URL.

auth.js

function getAuthenticationUrl() {
  return "/oauth2callback";
}

Let's fix this by setting up Google OAuth 2.0 authentication and redirecting the user to the real Google sign in screen.

Setup OAuth 2.0 client

First, you need to create a web application client for authentication.

Then, copy the URL that you've been using to access the app from the browser. This should be of the form

https://8080-dot-<9 digit number>-dot-devshell.appspot.com/ 

Use the following steps to configure your OAuth 2.0 setup:

  1. Visit Google Cloud Console. If necessary, select the project you created earlier from the Choose Project dropdown.
  2. From the left navigation, click API Manager > Credentials.
  3. Click Create Credentials > OAuth client ID.
  4. Click the Configure consent screen button.
  5. Choose an email address.
  6. Configure a Product name which identifies your application to users when they sign in
  7. Click Save.
  8. Choose Web application for the Application type.
  9. Paste the URL you copied at the start of this section into Authorized redirect URIs and append /oauth2callback, for example:
    https://8080-dot-<9 digit number>-dot-devshell.appspot.com/oauth2callback
  10. Click Create.
  11. Make note of the Client ID and Client secret that are displayed. You'll need them for configuration later.

Update configuration

The node.js application needs to be configured to use the Client ID you created.

In the project directory, edit the config.js file and do the following:

  1. Replace the placeholder values for clientId and clientSecret
  2. Replace http://localhost:8080/oauth2callback with the URL that you copied at the start of the Setup OAuth 2.0 client section. Don't forget to include the /oauth2callback.

config.js

module.exports = {
  // ...
  oauth2: {
    clientId: '[Client ID for web application credentials]',
    clientSecret: '[Client Secret for web application credentials]',
    redirectUrl: process.env.REDIRECT_URL || 'http://localhost:8080/oauth2callback'
  }
};

Redirect to sign in screen

To begin, install the googleapis npm package, which you will use to generate the authentication URL and fetch profile information for the logged in user.

npm install googleapis --save

In the project directory, edit the auth.js file and add the following code to require the googleapis package:

auth.js

module.exports = function(config) {

  // TODO: define this variable below
  var googleapis = require('googleapis');

  function getAuthenticationUrl() {
    // ...

Now, replace the current getAuthenticationUrl function with the following:

auth.js

function getAuthenticationUrl() {
  var client = new googleapis.auth.OAuth2(
    config.oauth2.clientId,
    config.oauth2.clientSecret,
    config.oauth2.redirectUrl
  );  
  // Use 'profile' scope to authorize fetching the user's profile
  return client.generateAuthUrl({ scope: ['profile'] }); 
}

The clientId and clientSecret are read from the config.js file that you edited earlier.

Restart the node application and click ‘Sign in'. You should be redirected to the Google sign in page.

If you login, you will be redirected back to the sample application, but you will still appear to be signed in as "Fake User" because the callback is not implemented to fetch the user profile.

Let's fix that!

Fetch user profile

Enable the Google+ API so the application can call the API to fetch user profiles.

  1. Under API Manager > Library > Social APIs search for the "Google+ API"
  2. Click on Google+ API
  3. Click Enable

To fetch the profile of the authenticated user, the /oauth2callback route that Google OAuth 2.0 redirects to calls auth.getUser from auth.js, passing it the provided ?code query string that can be used to fetch the user's profile.

The returned user profile is stored in the application session (via cookies).

app.js

var session = require('cookie-session');
app.use(session({ signed: true, secret: config.cookieSecret }));
// ...

/* Use OAuth 2.0 authorization code to fetch user's profile */
app.get('/oauth2callback', function(req, res, next) {
  auth.getUser(req.query.code, function(err, user) {
    if (err) return next(err);
    req.session.user = user;
    res.redirect('/');
  }); 
});

Currently, auth.getUser always returns "Fake User."

auth.js

function getUser(authorizationCode, callback) {
  var error = null;
  var fakeUser = { id: 123, name: 'Fake User' };
  callback(error, fakeUser);
}

Let's fix this!

Replace the current getUser function with the following:

auth.js

function getUser(authorizationCode, callback) {
  var client = new googleapis.auth.OAuth2(
    config.oauth2.clientId,
    config.oauth2.clientSecret,
    config.oauth2.redirectUrl
  );  
  // With the code returned from OAuth flow, get an access token
  client.getToken(authorizationCode, function(err, tokens) {
    if (err) return callback(err);
    // Configure this Google API client to use the access token
    client.setCredentials(tokens);
    // Call the Google+ API to get the profile of the user who authenticated
    googleapis.plus('v1').people.get({ userId: 'me', auth: client }, function(err, profile) {
      if (err) return callback(err);
      var user = { 
        id: profile.id,
        name: profile.displayName,
        imageUrl: profile.image.url
      };  
      callback(null, user);
    }); 
  }); 
}

This code gets an access token for the user who authenticated and then calls the People.get method of the Google+ API using the user's access token.

The sample application expects a user object in the following format:

{
  id: <unique ID of user>,
  name: <display name>,
  imageUrl: <publicly accessible URL to user profile image>
}

Restart the node application the try and Sign in again (you may need to Sign out first).

You should see your name and profile image displayed!

Summary

In this step, you implemented an OAuth 2.0 web authentication flow and fetched the authenticated user's profile information.

Next up

Next, you will associate users with the books they add and query Datastore for the current user's books.

When a user is signed in, the sample application passes the ID of the signed in user to books.addBook when new books are created.

app.js

/* Add a new book */
app.post('/books', function(req, res, next) {
  // ...

  var userId;
  if (req.session.user)
    userId = req.session.user.id;

  books.addBook(req.body.title, req.body.author, coverImageData, userId, function(err) {
    if (err) return next(err);
    res.redirect(req.get('Referer') || '/');
  })
});

Currently, we're not tracking this userId when saving book entities.

To fix this, update the entity object in the books.addBook function of books.js to include the userId when provided:

books.js

  if (userId)
    entity.data.userId = userId;
function addBook(title, author, coverImageData, userId, callback) {
  var entity = {
    key: datastore.key('Book'),
    data: {
      title: title,
      author: author,
    }
  };
  
  // Add the code here


  if (coverImageData)
    // ...

Now, books created by authenticated users will be associated with the books they add. This means we can query Datastore to list only the books added by that user.

The sample application displays a My Library link when a user is signed in. When clicked, only books owned by the user are displayed.

A user's books are fetched by calling books.getUserBooks in books.js.

app.js

/* Fetch books created by the currently logged in user and display them */
app.get('/mine', function(req, res, next) {
  if (! req.session.user) return res.redirect('/');
  books.getUsersBooks(req.session.user.id, function(err, books) {
    if (err) return next(err);
    res.render('index', { books: books, user: req.session.user }); 
  }); 
});

Sign into the application and click on My Library. You will receive an error:

Let's fix that!

Currently, the getUserBooks function in books.js is a placeholder that simply returns an error:

books.js

function getUserBooks(userId, callback) {
  callback(new Error('books.getUserBooks [Not Yet Implemented]'));
}

In the project directory, edit the books.js file and replace the getUserBooks function with the following:

books.js

function getUserBooks(userId, callback) {
  var query = datastore.createQuery(['Book']).filter('userId', '=', userId);
  datastore.runQuery(query, callback);
}

Restart the node application, sign in, and click on the My Library link. This should now list only the books you've added!

If you haven't added any books, you will see "There are no books!". In this case, sign in (if you haven't already), add a book, and then click the ‘My Library' link again to view the list of only books you have added.

Summary

In this step, you associated books with the user who created them and queried books from Datastore, filtering them by the userId property.

You learned how to integrate Datastore and Cloud Storage into a node.js application and use OAuth 2.0 to authenticate users!

What we've covered

Next Steps

Give us your feedback

Watch these videos: