Migrate from App Engine Users service to Cloud Identity Platform (Module 21)

1. Overview

The Serverless Migration Station series of codelabs (self-paced, hands-on tutorials) and related videos aim to help Google Cloud serverless developers modernize their appications by guiding them through one or more migrations, primarily moving away from legacy services. Doing so makes your apps more portable and gives you more options and flexibility, enabling you to integrate with and access a wider range of Cloud products and more easily upgrade to newer language releases. While initially focusing on the earliest Cloud users, primarily App Engine (standard environment) developers, this series is broad enough to include other serverless platforms like Cloud Functions and Cloud Run, or elsewhere if applicable.

The purpose of this codelab is to show Python 2 App Engine developers how to migrate from App Engine Users API/service to Cloud Identity Platform (GCIP). There is also an implicit migration from App Engine NDB to Cloud NDB for Datastore access (primarily covered in Migration Module 2) as well as an upgrade to Python 3.

Module 20 covers how to add the use of the Users API to the Module 1 sample app. In this module, you will take the finished Module 20 app and migrate its usage to Cloud Identity Platform.

You'll learn how to

  • Replace the use of App Engine Users service with Cloud Identity Platform
  • Replace the use of App Engine NDB with Cloud NDB (also see Module 2)
  • Setup different authentication identity providers using Firebase Auth
  • Use the Cloud Resource Manager API to get project IAM information
  • Use the Firebase Admin SDK to get user information
  • Port the sample application to Python 3

What you'll need

Survey

How will you use this tutorial?

Only read through it Read it and complete the exercises

How would you rate your experience with Python?

Novice Intermediate Proficient

How would you rate your experience with using Google Cloud services?

Novice Intermediate Proficient

2. Background

The App Engine Users service is a user authentication system for use by App Engine apps. It provides Google Sign-In as its identity provider, provides convenient login and logout links for use in apps, and supports the concept of admin users and admin-only functionality. To improve application portability, Google Cloud recommends migrating from legacy App Engine bundled services to Cloud standalone services, for example, from the Users service to Cloud Identity Platform, amongst others.

Identity Platform is based on Firebase Authentication, and adds a number of enterprise features including multi-factor authentication, OIDC & SAML SSO support, multi-tenancy, 99.95% SLA, and more. These differences are also highlighted on the Identity Platform and Firebase Authentication product comparison page. Both products have significantly more features than the functionality provided by the Users service.

This Module 21 codelab demonstrates switching the app's user authentication from Users service to Identity Platform features that most closely mirrors the functionality demonstrated in Module 20. Module 21 also features a migration from App Engine NDB to Cloud NDB for Datastore access, repeating the Module 2 migration.

While the Module 20 code is "advertised" as a Python 2 sample app, the source itself is Python 2 and 3 compatible, and it remains that way even after migrating to Identity Platform (and Cloud NDB) here in Module 21. It's possible to keep using the Users service while upgrading to Python 3 as migrating to Identity Platform is optional. See the Module 17 codelab and video to learn how to continue using the bundled services while upgrading to 2nd-generation runtimes like Python 3.

This tutorial features the following steps:

  1. Setup/Prework
  2. Update configuration
  3. Modify application code

3. Setup/Prework

This section explains how to:

  1. Set up your Cloud project
  2. Get baseline sample app
  3. (Re)Deploy and validate baseline app
  4. Enable new Google Cloud services/APIs

These steps ensure you're starting with working code which is ready for migration to standalone Cloud services.

1. Setup project

If you completed the Module 20 codelab, reuse that same project (and code). Alternatively, create a brand new project or reuse another existing project. Ensure the project has an active billing account and an enabled App Engine app. Find your project ID and have it handy during this codelab and use it whenever you encounter the PROJ_ID variable.

2. Get baseline sample app

One of the prerequisites is a working Module 20 App Engine app, so either complete its codelab (recommended; link above) or copy the Module 20 code from the repo. Whether you use yours or ours, this is where we'll begin ("START"). This codelab walks you through the migration, concluding with code that resembles what's in the Module 21 repo folder ("FINISH").

Copy the Module 20 repo folder. It should look like the output below, and may possibly have a lib folder if you did the Module 20 codelab:

$ ls
README.md               appengine_config.py     templates
app.yaml                main.py                 requirements.txt

3. (Re)Deploy and validate baseline app

Execute the following steps to deploy the Module 20 app:

  1. Delete the lib folder if there is one and run pip install -t lib -r requirements.txt to repopulate it. You may need to use pip2 if you have both Python 2 and 3 installed.
  2. Ensure you have installed and initialized the gcloud command-line tool and reviewed its usage.
  3. If you don't want to enter your PROJ_ID with each gcloud command issued, set the Cloud project with gcloud config set project PROJ_ID first.
  4. Deploy the sample app with gcloud app deploy
  5. Confirm the app runs as expected without errors. If you've completed the Module 20 codelab, the app displays user login information (user email, possible "admin badge", and login/logout button) at the top along with the most recent visits (illustrated below).

907e64c19ef964f8.png

Signing-in as a regular user causes the user's email address to be displayed, and the "Login" button changes to a "Logout" button:

ad7b59916b69a035.png

Signing-in as an admin user causes the user's email address to be displayed along with "(admin)" next to it:

867bcb3334149e4.png

4. Enable new Google Cloud APIs/services

Introduction

The Module 20 app uses the App Engine NDB and Users APIs, bundled services which don't require additional setup, but standalone Cloud services do, and the updated app will employ both Cloud Identity Platform and Cloud Datastore (via the Cloud NDB client library). Furthermore, our need to determine App Engine admin users also requires the use of the Cloud Resource Manager API.

Cost

  • App Engine and Cloud Datastore have "Always Free" tier quotas, and so long as you stay under those limits, you shouldn't incur charges completing this tutorial. Also see the App Engine pricing page and the Cloud Datastore pricing page for more details.
  • Use of the Cloud Identity Platform is billed depending on the number of monthly active users (MAUs) or authentication verifications; some version of "free" is available for each usage model. See its pricing page for more details. Furthermore, while App Engine and Cloud Datastore require billing, use of GCIP by itself doesn't require enabling billing as long as you don't exceed its instrumentless daily quotas, so consider this for Cloud projects that don't involve billing-required Cloud APIs/services.
  • Use of the Cloud Resource Manager API is free for the most part per its pricing page.

Users enable Cloud APIs from the Cloud console or from the command-line (via the gcloud command, part of the Cloud SDK), depending on your preference. Let's start with the Cloud Datastore and Cloud Resource Manager APIs.

From the Cloud Console

Go to the API Manager's Library page (for the correct project) in the Cloud Console, and search for an API using the search bar. c7a740304e9d35b.png

Enable these APIs:

Find and click the Enable button for each API separately—you may be prompted for billing information. For example, here's the page for the Resource Manager API:

fc7bd8f4c49d12e5.png

The button changes to Manage when it has been enabled (generally after a few seconds):

8eca12d6cc7b45b0.png

Enable Cloud Datastore in the same way:

83811599b110e46b.png

From the command-line

While it is visually informative to enable APIs from the console, some prefer the command-line. You get the added bonus of being able to enable any number of APIs at once. Issue this command to enable both the Cloud Datastore and Cloud Resource Manager APIs and wait for the operation to complete, as illustrated here:

$ gcloud services enable cloudresourcemanager.googleapis.com datastore.googleapis.com
Operation "operations/acat.p2-aaa-bbb-ccc-ddd-eee-ffffff" finished successfully.

You may be prompted for billing information.

The "URLs" for each API used in the command above are called the API service names, and they can be found at the bottom of the library page for each API. If you wish to enable other Cloud APIs for your own apps, you can find their respective service names on their corresponding API pages. This command lists all service names for APIs you can enable:

gcloud services list --available --filter="name:googleapis.com".

Whether in the Cloud console or on the command-line, once you've completed the steps above, our sample is now able to access those APIs. The next steps are to enable the Cloud Identity Platform and make the necessary code changes.

Enable and setup Cloud Identity Platform (Cloud console only)

Cloud Identity Platform is a Marketplace service because it connects to or depends on a resource outside of Google Cloud, for example, Firebase Authentication. At this time, you can only enable Marketplace services from the Cloud console. Follow the steps below:

  1. Go to the Cloud Identity Platform page in the Cloud Marketplace and click the Enable button there. Upgrade from Firebase Authentication if prompted—doing so unlocks additional features, such as those described earlier in the Background section. Here is the Marketplace page highlighting the Enable button: 28475f1c9b29de69.png
  2. Once Identity Platform is enabled, you may be taken automatically to the Identity Providers page. If not, use this convenient link to get there. fc2d92d42a5d1dd7.png
  3. Enable the Google Auth provider. If no providers have been set up, click Add a Provider and select Google. When you return to this screen, the Google entry should be enabled. Google is the only auth provider we're using in this tutorial to mirror the App Engine Users service as a lightweight Google Sign-In service. In your own apps, you can enable additional auth providers.
  4. When you've selected and set up Google and other desired auth providers, click Application Setup Details, and from the ensuring dialog window, copy the apiKey and authDomain in the config object on Web tab, saving both of them somewhere safe. Why not copy all of it? The snippet in this dialog is hardcoded and dated, so just save the most important bits and use them in our code with more concurrent Firebase Auth usage. Once you've copied the values and saved them somewhere safe, click the Close button, completing all of the necessary setup. bbb09dcdd9be538e.png

4. Update configuration

Updates in configuration include both changing various configuration files as well as creating the equivalent of App Engine but within the Cloud Identity Platform ecosystem.

appengine_config.py

  • If upgrading to Python 3, delete appengine_config.py
  • If planning on modernizing to Identity Platform but staying on Python 2, don't delete the file. Instead, we will update it later during the Python 2 backport.

requirements.txt

Module 20's requirements.txt file only listed Flask. For Module 21, add the following packages:

The contents of requirements.txt should now look like this:

flask
google-auth
google-cloud-ndb
google-cloud-resource-manager
firebase-admin

app.yaml

  • Upgrading to Python 3 means simplifying the app.yaml file. Remove everything except for the runtime directive, and set that to a currently supported version of Python 3. The example currently uses version 3.10.
  • If you're staying with Python 2, take no action here yet.

BEFORE:

runtime: python27
threadsafe: yes
api_version: 1

handlers:
- url: /.*
  script: main.app

The Module 20 sample app doesn't have static file handlers. If your apps do, leave them intact. You can remove all your script handlers if desired or just leave them there for reference as long as you change their handles to auto, as described in the app.yaml migration guide. With these changes, the updated app.yaml for Python 3 is simplified to:

AFTER:

runtime: python310

Other configuration updates

Whether staying on Python 2 or porting to Python 3, if you have a lib folder, delete it.

5. Modify application code

This section features updates to the main application file, main.py, replacing use of App Engine Users service with Cloud Identity Platform. After updating the main application, you'll update the web template, templates/index.html.

Update imports and initialization

Follow the steps below for updating the imports and initializing application resources:

  1. For the imports, replace App Engine NDB with Cloud NDB.
  2. Along with Cloud NDB, also import Cloud Resource Manager.
  3. Identity Platform is based on Firebase Auth, so import the Firebase Admin SDK.
  4. Cloud APIs require use of an API client, so initiate it for Cloud NDB just below initializing Flask.

While the Cloud Resource Manager package is imported here, we'll use it at a later stage in app initialization. Below are the imports and initialization from Module 20 followed by how the sections should look after the implementing the changes above:

BEFORE:

from flask import Flask, render_template, request
from google.appengine.api import users
from google.appengine.ext import ndb

app = Flask(__name__)

AFTER:

from flask import Flask, render_template, request
from google.auth import default
from google.cloud import ndb, resourcemanager
from firebase_admin import auth, initialize_app

# initialize Flask and Cloud NDB API client
app = Flask(__name__)
ds_client = ndb.Client()

Support for App Engine Admin users

There are two components to add to the app that supports the recognition of admin users:

  • _get_gae_admins() — collates set of admin users; called once and saved
  • is_admin() — checks if signed-in user is an admin user; called on any user login

The utility function, _get_gae_admins(), calls the Resource Manager API to fetch the current Cloud IAM allow-policy. The allow-policy defines and enforces what roles are granted to which principals (human users, service accounts, etc.). The setup includes:

  • Fetching the Cloud project ID (PROJ_ID)
  • Creating a Resource Manager API client (rm_client)
  • Creating a (read-only) set of App Engine Admin roles (_TARGETS)

The Resource Manager requires the Cloud project ID, so import google.auth.default() and call that function to get the project ID. That call features a parameter that looks like a URL but is an OAuth2 permission scope. When running apps in the cloud, for example, on a Compute Engine VM or App Engine app, a default service account is provided which has broad privileges. In keeping with the best practice of least privilege, we recommend creating your own user-managed service accounts.

For API calls, it's best to further reduce the scope of your apps to a minimum level needed to function properly. The Resource Manager API call we'll be making is get_iam_policy() which needs one of the following scopes to operate:

  • https://www.googleapis.com/auth/cloud-platform
  • https://www.googleapis.com/auth/cloud-platform.read-only
  • https://www.googleapis.com/auth/cloudplatformprojects
  • https://www.googleapis.com/auth/cloudplatformprojects.readonly

The sample app only needs read-only access to the allow-policy. It doesn't modify the policy nor does it need access to the entire project. That means the app doesn't need any of the first three permissions needed. The last one is all that's required, and that's what we're implementing for the sample app.

The main body of the function creates an empty set of admin users (admins), fetches the allow_policy via get_iam_policy(), and loops through all of its bindings looking specifically for App Engine Admin roles:

  • roles/viewer
  • roles/editor
  • roles/owner
  • roles/appengine.appAdmin

For each target role found, it collates which users belong to that role, adding them to the overall set of admin users. It ends by returning all of the admin users found and cached as a constant (_ADMINS) for the life of this App Engine instance. We'll see that call coming up shortly.

Add the following _get_gae_admins() function definition to main.py just below instantiating the Cloud NDB API client (ds_client):

def _get_gae_admins():
    'return set of App Engine admins'
    # setup constants for calling Cloud Resource Manager API
    _, PROJ_ID = default(  # Application Default Credentials and project ID
            ['https://www.googleapis.com/auth/cloudplatformprojects.readonly'])
    rm_client = resourcemanager.ProjectsClient()
    _TARGETS = frozenset((     # App Engine admin roles
            'roles/viewer',
            'roles/editor',
            'roles/owner',
            'roles/appengine.appAdmin',
    ))

    # collate users who are members of at least one GAE admin role (_TARGETS)
    admins = set()                      # set of all App Engine admins
    allow_policy = rm_client.get_iam_policy(resource='projects/%s' % PROJ_ID)
    for b in allow_policy.bindings:     # bindings in IAM allow-policy
        if b.role in _TARGETS:          # only look at GAE admin roles
            admins.update(user.split(':', 1).pop() for user in b.members)
    return admins

When the users login to the app, the following occurs:

  1. A quick check is made from the web template after a user signs into Firebase.
  2. When the auth state changes in the template, an Ajax-style fetch() call is made to /is_admin whose handler is the next function, is_admin().
  3. The Firebase ID Token is passed in the POST body to is_admin(), which grabs it out of the headers and calls the Firebase Admin SDK to validate it. If it's a valid user, extract their email address and check if it's an admin user.
  4. The Boolean result is then returned to the template as a successful 200.

Add is_admin() to main.py just after _get_gae_admins():

@app.route('/is_admin', methods=['POST'])
def is_admin():
    'check if user (via their Firebase ID token) is GAE admin (POST) handler'
    id_token = request.headers.get('Authorization')
    email = auth.verify_id_token(id_token).get('email')
    return {'admin': email in _ADMINS}, 200

All of the code from both functions are required to replicate the functionality available from the Users service, specifically its is_current_user_admin() function. This function call in Module 20 did all the heavy-lifting, unlike Module 21 where we implement a replacement solution. The good news is that the app is no longer dependent on an App Engine-only service, meaning you can move your apps to Cloud Run or other services. Furthermore, you can also change the definition of "admin user" for your own apps just by switching to the desired roles in _TARGETS whereas the Users service is hardcoded for the App Engine admin roles.

Initialize Firebase Auth and cache App Engine admin users

We could have initialized Firebase Auth at the top near the same spot the Flask app is initialized and Cloud NDB API client created, but there was no need to until all of the admin code had been defined, which is where we are now. Similarly, now that _get_gae_admins() is defined, call it to cache the list of admin users.

Add these lines just under the function body of is_admin():

# initialize Firebase and fetch set of App Engine admins
initialize_app()
_ADMINS = _get_gae_admins()

Visit data model updates

The Visit data model doesn't change. Datastore access requires explicit use of the Cloud NDB API client context manager, ds_client.context(). In code, this means you wrap Datastore calls in both store_visit() and fetch_visits() inside Python with blocks. This update is identical to Module 2. Make the changes as follows:

BEFORE:

class Visit(ndb.Model):
    'Visit entity registers visitor IP address & timestamp'
    visitor   = ndb.StringProperty()
    timestamp = ndb.DateTimeProperty(auto_now_add=True)

def store_visit(remote_addr, user_agent):
    'create new Visit entity in Datastore'
    Visit(visitor='{}: {}'.format(remote_addr, user_agent)).put()

def fetch_visits(limit):
    'get most recent visits'
    return Visit.query().order(-Visit.timestamp).fetch(limit)

AFTER:

class Visit(ndb.Model):
    'Visit entity registers visitor IP address & timestamp'
    visitor   = ndb.StringProperty()
    timestamp = ndb.DateTimeProperty(auto_now_add=True)

def store_visit(remote_addr, user_agent):
    'create new Visit entity in Datastore'
    with ds_client.context():
        Visit(visitor='{}: {}'.format(remote_addr, user_agent)).put()

def fetch_visits(limit):
    'get most recent visits'
    with ds_client.context():
        return Visit.query().order(-Visit.timestamp).fetch(limit)

Move user login logic to web template

The App Engine Users service is server-side whereas Firebase Auth and Cloud Identity Platform are predominantly client-side. As a result, much of the user management code in the Module 20 app moves to the Module 21 web template.

In main.py, the web context passes five essential pieces of data to the template, the first four listed are tied to user management and differ depending on whether the user is signed-in or not:

  • who — user's email if signed-in or user otherwise
  • admin — (admin) badge if signed-in user is an admin
  • sign — show Login or Logout button
  • link — sign-in or sign-in out links on button click
  • visits — most recent visits

BEFORE:

@app.route('/')
def root():
    'main application (GET) handler'
    store_visit(request.remote_addr, request.user_agent)
    visits = fetch_visits(10)

    # put together users context for web template
    user = users.get_current_user()
    context = {  # logged in
        'who':   user.nickname(),
        'admin': '(admin)' if users.is_current_user_admin() else '',
        'sign':  'Logout',
        'link':  '/_ah/logout?continue=%s://%s/' % (
                      request.environ['wsgi.url_scheme'],
                      request.environ['HTTP_HOST'],
                  ),  # alternative to users.create_logout_url()
    } if user else {  # not logged in
        'who':   'user',
        'admin': '',
        'sign':  'Login',
        'link':  users.create_login_url('/'),
    }

    # add visits to context and render template
    context['visits'] = visits  # display whether logged in or not
    return render_template('index.html', **context)

All of the user management is moving to the web template, so we're left with just the visits, bringing the main handler back to what we had all the way back in the Module 1 app:

AFTER:

@app.route('/')
def root():
    'main application (GET) handler'
    store_visit(request.remote_addr, request.user_agent)
    visits = fetch_visits(10)
    return render_template('index.html', visits=visits)

Update web template

What do all the updates from the previous section look like in the template? Mainly moving user management from the app to Firebase Auth running in the template and a partial port of all that code we moved into JavaScript. We saw main.py shrink quite a bit, so expect similar growth in templates/index.html.

BEFORE:

<!doctype html>
<html>
<head>
<title>VisitMe Example</title>
</head>
<body>
<p>
Welcome, {{ who }} <code>{{ admin }}</code>
<button id="logbtn">{{ sign }}</button>
</p><hr>

<h1>VisitMe example</h1>
<h3>Last 10 visits</h3>
<ul>
{% for visit in visits %}
    <li>{{ visit.timestamp.ctime() }} from {{ visit.visitor }}</li>
{% endfor %}
</ul>

<script>
document.getElementById("logbtn").onclick = () => {
    window.location.href = '{{ link }}';
};
</script>
</body>
</html>

Replace the entire web template with the contents below:

AFTER:

<!doctype html>
<html>
<head>
<title>VisitMe Example</title>

<script type="module">
// import Firebase module attributes
import {
        initializeApp
} from "https://www.gstatic.com/firebasejs/9.10.0/firebase-app.js";
import {
        GoogleAuthProvider,
        getAuth,
        onAuthStateChanged,
        signInWithPopup,
        signOut
} from "https://www.gstatic.com/firebasejs/9.10.0/firebase-auth.js";

// Firebase config:
// 1a. Go to: console.cloud.google.com/customer-identity/providers
// 1b. May be prompted to enable GCIP and upgrade from Firebase
// 2. Click: "Application Setup Details" button
// 3. Copy: 'apiKey' and 'authDomain' from 'config' variable
var firebaseConfig = {
        apiKey: "YOUR_API_KEY",
        authDomain: "YOUR_AUTH_DOMAIN",
};

// initialize Firebase app & auth components
initializeApp(firebaseConfig);
var auth = getAuth();
var provider = new GoogleAuthProvider();
//provider.setCustomParameters({prompt: 'select_account'});

// define login and logout button functions
function login() {
    signInWithPopup(auth, provider);
};

function logout() {
    signOut(auth);
};

// check if admin & switch to logout button on login; reset everything on logout
onAuthStateChanged(auth, async (user) => {
    if (user && user != null) {
        var email = user.email;
        who.innerHTML = email;
        logbtn.onclick = logout;
        logbtn.innerHTML = "Logout";
        var idToken = await user.getIdToken();
        var rsp = await fetch("/is_admin", {
                method: "POST",
                headers: {Authorization: idToken}
        });
        var data = await rsp.json();
        if (data.admin) {
            admin.style.display = "inline";
        }
    } else {
        who.innerHTML = "user";
        admin.style.display = "none";
        logbtn.onclick = login;
        logbtn.innerHTML = "Login";
    }
});
</script>
</head>

<body>
<p>
Welcome, <span id="who"></span> <span id="admin"><code>(admin)</code></span>
<button id="logbtn"></button>
</p><hr>

<h1>VisitMe example</h1>
<h3>Last 10 visits</h3>
<ul>
{% for visit in visits %}
    <li>{{ visit.timestamp.ctime() }} from {{ visit.visitor }}</li>
{% endfor %}
</ul>

<script>
var who    = document.getElementById("who");
var admin  = document.getElementById("admin");
var logbtn = document.getElementById("logbtn");
</script>
</body>
</html>

There are many components in this HTML body, so let's take them one piece at a time.

Firebase imports

While still in the header of the HTML document, once past the page title, import the Firebase components needed. Firebase components are now broken into multiple modules for efficiency. The code to initialize Firebase is imported from the main Firebase app module while functions that manage Firebase auth, Google as an auth provider, signing in and out, and auth state change "callback" are all imported from the Firebase Auth module:

<!doctype html>
<html>
<head>
<title>VisitMe Example</title>

<script type="module">
// import Firebase module attributes
import {
        initializeApp
} from "https://www.gstatic.com/firebasejs/9.10.0/firebase-app.js";
import {
        GoogleAuthProvider,
        getAuth,
        onAuthStateChanged,
        signInWithPopup,
        signOut
} from "https://www.gstatic.com/firebasejs/9.10.0/firebase-auth.js";

Firebase configuration

Earlier during the Identity Platform setup part of this tutorial, you saved the apiKey and authDomain from the Application Setup Details dialog. Add those values to the firebaseConfig variable in this next section. A link to more details instructions is provided in the comments:

// Firebase config:
// 1a. Go to: console.cloud.google.com/customer-identity/providers
// 1b. May be prompted to enable GCIP and upgrade from Firebase
// 2. Click: "Application Setup Details" button
// 3. Copy: 'apiKey' and 'authDomain' from 'config' variable
var firebaseConfig = {
        apiKey: "YOUR_API_KEY",
        authDomain: "YOUR_AUTH_DOMAIN",
};

Firebase initialization

The next section initializes Firebase with this configuration information.

// initialize Firebase app & auth components
initializeApp(firebaseConfig);
var auth = getAuth();
var provider = new GoogleAuthProvider();
//provider.setCustomParameters({prompt: 'select_account'});

This sets the ability to use Google as an auth provider, and provides a commented-out option for showing the account selector even if there's only one Google account registered in your browser session. In other words, when you have multiple accounts, you're presented with this "account-picker" as expected: a38369389b7c4c7e.png However, if there's only one user in the session, the login process completes automatically without any user interaction. (The popup appears then disappears.) You can force the account-picker dialog to show up for one user (vs. immediately logging into the app) by uncommenting the custom parameter line. If enabled, even single-user logins bring up the account-picker: b75624cb68d94557.png

Login and logout functions

The next lines of code make up the functions for the login or logout button clicks:

// define login and logout button functions
function login() {
    signInWithPopup(auth, provider);
};

function logout() {
    signOut(auth);
};

Sign-in and sign-out actions

The last major section in this <script> block is the function that's called for every auth change (sign-in or sign-out).

// check if admin & switch to logout button on login; reset everything on logout
onAuthStateChanged(auth, async (user) => {
    if (user && user != null) {
        var email = user.email;
        who.innerHTML = email;
        logbtn.onclick = logout;
        logbtn.innerHTML = "Logout";
        var idToken = await user.getIdToken();
        var rsp = await fetch("/is_admin", {
                method: "POST",
                headers: {Authorization: idToken}
        });
        var data = await rsp.json();
        if (data.admin) {
            admin.style.display = "inline";
        }
    } else {
        who.innerHTML = "user";
        admin.style.display = "none";
        logbtn.onclick = login;
        logbtn.innerHTML = "Login";
    }
});
</script>
</head>

The code in Module 20 determining whether to send a "user logged in" template context vs. a "user logged out" context is transitioned here. The conditional at the top results in true if the user logged in successfully, triggering the following actions:

  1. The user's email address is set for display.
  2. The Login button changes to Logout.
  3. An Ajax-style call to /is_admin is made to determine whether to show the (admin) admin user badge.

When the user logs out, the else clause is executed to reset all the user information:

  1. Username set to user
  2. Any admin badge removed
  3. Logout button changed back to Login

Template variables

After the header section ends, the main body begins with the template variables that are replaced by HTML elements that change as necessary:

  1. Displayed user name
  2. (admin) admin badge (if applicable)
  3. Login or Logout button
<body>
<p>
Welcome, <span id="who"></span> <span id="admin"><code>(admin)</code></span>
<button id="logbtn"></button>
</p><hr>

Most recent visits and HTML element variables

The most recent visits code doesn't change, and the final <script> block sets the variables for the HTML elements that change for sign-in and sign-out listed just above:

<h1>VisitMe example</h1>
<h3>Last 10 visits</h3>
<ul>
{% for visit in visits %}
    <li>{{ visit.timestamp.ctime() }} from {{ visit.visitor }}</li>
{% endfor %}
</ul>

<script>
var who    = document.getElementById("who");
var admin  = document.getElementById("admin");
var logbtn = document.getElementById("logbtn");
</script>
</body>
</html>

This concludes the changes needed in the application and web template to switch from App Engine NDB and Users APIs to Cloud NDB and Identity Platform as well as upgrade to Python 3. Congratulations for arriving at your new Module 21 sample app! Our version is available for review in the Module 21b repo folder.

The next part of the codelab is optional (*) and only for users whose apps must remain on Python 2, leading you through the steps necessary to arrive at a working Python 2 Module 21 app.

6. *Python 2 backport

This optional section is for developers performing an Identity Platform migration but who must continue to run on the Python 2 runtime. If this is not a concern for you, skip this section.

To create a working Python 2 version of the Module 21 app, you need the following:

  1. Runtime requirements: Configuration files that support Python 2, and required changes in the main application to avoid Python 3 incompatibilities
  2. Minor library change: Python 2 was deprecated before some required features were added to the Resource Manager client library. As a result, you need an alternative way to access that missing functionality.

Let's take those steps now, starting with configuration.

Restore appengine_config.py

Earlier in this tutorial, you were guided to delete appengine_config.py since it's not used by the Python 3 App Engine runtime. For Python 2, not only must it be preserved, but the Module 20 appengine_config.py needs to be updated to support use of built-in 3rd-party libraries, namely grpcio and setuptools. Those packages are required whenever your App Engine app uses Cloud client libraries like those for Cloud NDB and Cloud Resource Manager.

You'll add those packages to app.yaml momentarily, but for your app to access them, the pkg_resources.working_set.add_entry() function from setuptools must be called. This allows copied (self-bundled or vendored) 3rd-party libraries installed in the lib folder to be able to communicate with built-in libraries.

Implement the following updates to your appengine_config.py file to effect these changes:

BEFORE:

from google.appengine.ext import vendor

# Set PATH to your libraries folder.
PATH = 'lib'
# Add libraries installed in the PATH folder.
vendor.add(PATH)

This code alone does not suffice to support the use of setuptools and grpcio. A few more lines are needed, so update appengine_config.py so it looks like this:

AFTER:

import pkg_resources
from google.appengine.ext import vendor

# Set PATH to your libraries folder.
PATH = 'lib'
# Add libraries installed in the PATH folder.
vendor.add(PATH)
# Add libraries to pkg_resources working set to find the distribution.
pkg_resources.working_set.add_entry(PATH)

More details on changes required to support Cloud client libraries can be found in the migrating bundled services documentation.

app.yaml

Similar to appengine_config.py, the app.yaml file must be reverted to one that supports Python 2. Let's start with the original Module 20 app.yaml:

BEFORE:

runtime: python27
threadsafe: yes
api_version: 1

handlers:
- url: /.*
  script: main.app

In addition to setuptools and grpcio as mentioned previously, there is a dependency (not explicitly related to the Identity Platform migration) requiring the use of the Cloud Storage client library, and that needs another built-in 3rd-party package, ssl. Add all three in a new libraries section, selecting the "latest" available versions of those packages, to app.yaml:

AFTER:

runtime: python27
threadsafe: yes
api_version: 1

handlers:
- url: /.*
  script: main.app

libraries:
- name: grpcio
  version: latest
- name: setuptools
  version: latest
- name: ssl
  version: latest

requirements.txt

For Module 21, we added Google Auth, Cloud NDB, Cloud Resource Manager, and Firebase Admin SDK to the Python 3 requirements.txt. The situation for Python 2 is more complex:

  • The Resource Manager API provides the allow-policy functionality needed for the sample app. Unfortunately this support wasn't yet available in the final Python 2 version of the Cloud Resource Manager client library. (It is only available in the Python 3 version.)
  • As a result, an alternative way to access this feature from the API is required. The solution is to use the lower-level Google APIs client library to communicate with the API. To switch to this client library, replace google-cloud-resource-manager with the lower-level google-api-python-client package.
  • Because Python 2 has been sunset, the dependency graph supporting Module 21 requires locking certain packages to specific versions. Some packages must be called out even if they aren't specified in the Python 3 app.yaml.

BEFORE:

flask

Starting with the Module 20 requirements.txt, update it to the following for a working Module 21 app:

AFTER:

grpcio==1.0.0
protobuf<3.18.0
six>=1.13.0
flask
google-gax<0.13.0
google-api-core==1.31.1
google-api-python-client<=1.11.0
google-auth<2.0dev
google-cloud-datastore==1.15.3
google-cloud-firestore==1.9.0
google-cloud-ndb
google-cloud-pubsub==1.7.0
firebase-admin

The package and version numbers will be updated in the repo as dependencies change, but this app.yaml suffices for a functioning app at the time of this writing.

Other configuration updates

If you haven't deleted the lib folder from earlier in this codelab, do so now. With the newly-updated requirements.txt, issue this familiar command to install these requirements into lib:

pip install -t lib -r requirements.txt  # or pip2

If you have both Python 2 and 3 installed on your development system, you may need to use pip2 instead of pip.

Modify application code

Fortunately, most of the required changes are in the configuration files. The only change needed in application code is a minor update to use the lower-level Google API client library instead of the Resource Manager client library to access the API. There are no updates required to the templates/index.html web template.

Update imports and initialization

Replace the Resource Manager client library (google.cloud.resourcemanager) with the Google APIs client library (googleapiclient.discovery), as illustrated below:

BEFORE:

from flask import Flask, render_template, request
from google.auth import default
from google.cloud import ndb, resourcemanager
from firebase_admin import auth, initialize_app

AFTER:

from flask import Flask, render_template, request
from google.auth import default
from google.cloud import ndb
from googleapiclient import discovery
from firebase_admin import auth, initialize_app

Support for App Engine Admin users

A few changes are needed in _get_gae_admins() to support use of the lower-level client library. Let's discuss what's changing first then give you all the code to update.

The Python 2 code requires use of both the credentials and the project ID returned from google.auth.default(). The credentials aren't used in Python 3, so it was assigned to a generic underscore ( _ ) dummy variable. Since it's needed for the Python 2 version, change the underscore to CREDS. Also, rather than creating a Resource Manager API client, you'll create an API service endpoint, similar in concept to an API client, so we're keeping the same variable name (rm_client). One difference is that instantiating a service endpoint requires credentials (CREDS).

These changes are reflected in the code below:

BEFORE:

_, PROJ_ID = default(  # Application Default Credentials and project ID
        ['https://www.googleapis.com/auth/cloudplatformprojects.readonly'])
rm_client = resourcemanager.ProjectsClient()

AFTER:

CREDS, PROJ_ID = default(  # Application Default Credentials and project ID
        ['https://www.googleapis.com/auth/cloud-platform'])
rm_client = discovery.build('cloudresourcemanager', 'v1', credentials=CREDS)

The other difference is that the Resource Manager client library returns allow-policy objects which use dotted-attribute notation while the lower-level client library returns Python dictionaries where square brackets ( [ ] ) are used, for example, use binding.role for the Resource Manager client library versus binding['role'] for the lower-level library. The former also uses "underscore_separated" names versus the lower-level library preferring "CamelCased" names plus a slightly different way to pass in API parameters.

These usage differences are shown below:

BEFORE:

allow_policy = rm_client.get_iam_policy(resource='projects/%s' % PROJ_ID)
for b in allow_policy.bindings:     # bindings in IAM allow-policy
    if b.role in _TARGETS:          # only look at GAE admin roles
        admins.update(user.split(':', 1).pop() for user in b.members)

AFTER:

allow_policy = rm_client.projects().getIamPolicy(resource=PROJ_ID).execute()
for b in allow_policy['bindings']:  # bindings in IAM allow-policy
    if b['role'] in _TARGETS:       # only look at GAE admin roles
        admins.update(user.split(':', 1).pop() for user in b['members'])

Putting all these changes together, replace the Python 3 _get_gae_admins() with this equivalent Python 2 version:

def _get_gae_admins():
    'return set of App Engine admins'
    # setup constants for calling Cloud Resource Manager API
    CREDS, PROJ_ID = default(  # Application Default Credentials and project ID
            ['https://www.googleapis.com/auth/cloud-platform'])
    rm_client = discovery.build('cloudresourcemanager', 'v1', credentials=CREDS)
    _TARGETS = frozenset((     # App Engine admin roles
            'roles/viewer',
            'roles/editor',
            'roles/owner',
            'roles/appengine.appAdmin',
    ))

    # collate users who are members of at least one GAE admin role (_TARGETS)
    admins = set()                      # set of all App Engine admins
    allow_policy = rm_client.projects().getIamPolicy(resource=PROJ_ID).execute()
    for b in allow_policy['bindings']:  # bindings in IAM allow-policy
        if b['role'] in _TARGETS:       # only look at GAE admin roles
            admins.update(user.split(':', 1).pop() for user in b['members'])
    return admins

The is_admin() function doesn't require any updates because it relies on _get_gae_admins() which has already been updated.

This concludes the changes required to backport the Python 3 Module 21 app to Python 2. Congratulations for arriving at your updated Module 21 sample app! You'll find all the code in the Module 21a repo folder.

7. Summary/Cleanup

The last steps in the codelab are to ensure principals (users or service accounts) running this app have the proper permissions to do so, then deploy your app to confirm it works as intended and the changes are reflected in the output.

Ability to read IAM allow-policy

Earlier, we introduced you to the four roles required to be recognized as an App Engine admin user, but there is now a fifth to become familiar with:

  • roles/viewer
  • roles/editor
  • roles/owner
  • roles/appengine.appAdmin
  • roles/resourcemanager.projectIamAdmin (for principals accessing the IAM allow-policy)

The roles/resourcemanager.projectIamAdmin role enables principals to determine whether an end-user is a member of any of the App Engine admin roles. Without membership in roles/resourcemanager.projectIamAdmin, calls to the Cloud Resource Manager API to get the allow-policy will fail.

You do not need to take any explicit action here as your app will run under App Engine's default service account which is automatically granted membership in this role. Even if you use the default service account during the development phase, we strongly recommend creating and using a user-managed service account with the minimal permissions required for your app to function properly. To grant membership to such a service account, run the following command:

$ gcloud projects add-iam-policy-binding PROJ_ID --member="serviceAccount:USR_MGD_SVC_ACCT@PROJ_ID.iam.gserviceaccount.com" --role=roles/resourcemanager.projectIamAdmin

PROJ_ID is the Cloud project ID and USR_MGD_SVC_ACCT@PROJ_ID.iam.gserviceaccount.com is the user-managed service account you create for your app. This command outputs the updated IAM policy for your project where you can confirm the service account has membership in roles/resourcemanager.projectIamAdmin. For more information, see the reference documentation. To repeat, you don't need to issue that command in this codelab, but save this as a reference for modernizing your own apps.

Deploy and verify application

Upload your app to the cloud with the standard gcloud app deploy command. Once deployed, you should see functionality almost identical to the Module 20 app except that you've successfully replaced the App Engine Users service with the Cloud Identity Platform (and Firebase Auth) for user management:

3a83ae745121d70.png

One difference you'll notice compared to Module 20 is that clicking on the Login results in a popup instead of a redirect, captured in some of the screenshots below. Like Module 20 however, the behavior differs slightly depending on how many Google accounts have been registered with the browser.

If there are no users registered with the browser or a single user who hasn't signed in yet, a generic Google Sign-in popup comes up:

8437f5f3d489a942.png

If a single user is registered with your browser but signs-in elsewhere, no dialog appears (or it pops up and closes immediately), and the app goes into a signed-in state (displays the user email and Logout button).

Some developers may want to provide an account-picker, even for a single user:

b75624cb68d94557.png

To implement this, uncomment the provider.setCustomParameters({prompt: 'select_account'}); line in the web template as described earlier.

If there are multiple users, the account-picker dialog pops up (see below). If not signed-in yet, the user will be prompted. If already signed-in, the popup disappears, and the app goes into a signed-in state.

c454455b6020d5e4.png

The signed-in state of Module 21 looks identical to Module 20's user interface:

49ebe4dcc1eff11f.png

The same is true for when an admin user has signed-in:

44302f35b39856eb.png

Unlike Module 21, Module 20 always accesses the logic for the web template content from the app (server-side code). A flaw of the Module 20 is that one visit is registered when the end-user hits the app the first time, and another one is registered when a user signs-in.

For Module 21, the login logic takes place in just the web template (client-side code). There is no required server-side trip to determine what content to display. The only call made to the server is the check for admin users after an end-user signs-in. This means that logins and logouts don't register additional visits, so the most recent visits list stays constant for user management actions. Notice the screenshots above display the same set of four visits across multiple user logins.

The Module 20 screenshots demonstrate the "double-visit bug" at the beginning of this codelab. Separate visits logs are displayed for each sign-in or sign-out action. Check the timestamps of the most recent visit for each screenshot displaying the chronological ordering.

Clean up

General

If you are done for now, we recommend you disable your App Engine app to avoid incurring billing. However if you wish to test or experiment some more, the App Engine platform has a free quota, and so as long as you don't exceed that usage tier, you shouldn't be charged. That's for compute, but there may also be charges for relevant App Engine services, so check its pricing page for more information. If this migration involves other Cloud services, those are billed separately. In either case, if applicable, see the "Specific to this codelab" section below.

For full disclosure, deploying to a Google Cloud serverless compute platform like App Engine incurs minor build and storage costs. Cloud Build has its own free quota as does Cloud Storage. Storage of that image uses up some of that quota. However, you might live in a region that does not have such a free tier, so be aware of your storage usage to minimize potential costs. Specific Cloud Storage "folders" you should review include:

  • console.cloud.google.com/storage/browser/LOC.artifacts.PROJECT_ID.appspot.com/containers/images
  • console.cloud.google.com/storage/browser/staging.PROJECT_ID.appspot.com
  • The storage links above depend on your PROJECT_ID and LOCation, for example, "us" if your app is hosted in the USA.

On the other hand, if you're not going to continue with this application or other related migration codelabs and want to delete everything completely, shut down your project.

Specific to this codelab

The services listed below are unique to this codelab. Refer to each product's documentation for more information:

Next steps

Beyond this tutorial, other migration modules that focus on moving away from the legacy bundled services to consider include:

  • Module 2: migrate from App Engine ndb to Cloud NDB
  • Modules 7-9: migrate from App Engine Task Queue (push tasks) to Cloud Tasks
  • Modules 12-13: migrate from App Engine Memcache to Cloud Memorystore
  • Modules 15-16: migrate from App Engine Blobstore to Cloud Storage
  • Modules 18-19: migrate from App Engine Task Queue (pull tasks) to Cloud Pub/Sub

App Engine is no longer the only serverless platform in Google Cloud. If you have a small App Engine app or one that has limited functionality and wish to turn it into a standalone microservice, or you want to break-up a monolithic app into multiple reusable components, these are good reasons to consider moving to Cloud Functions. If containerization has become part of your application development workflow, particularly if it consists of a CI/CD (continuous integration/continuous delivery or deployment) pipeline, consider migrating to Cloud Run. These scenarios are covered by the following modules:

  • Migrate from App Engine to Cloud Functions: see Module 11
  • Migrate from App Engine to Cloud Run: see Module 4 to containerize your app with Docker, or Module 5 to do it without containers, Docker knowledge, or Dockerfiles

Switching to another serverless platform is optional, and we recommend considering the best options for your apps and use cases before making any changes.

Regardless of which migration module you consider next, all Serverless Migration Station content (codelabs, videos, source code [when available]) can be accessed at its open source repo. The repo's README also provides guidance on which migrations to consider and any relevant "order" of Migration Modules.

8. Additional resources

Listed below are additional resources for developers exploring this or related migration modules further. Below, you can provide feedback on this content, find links to the code, and various pieces of documentation you may find useful.

Codelabs issues/feedback

If you find any issues with this codelab, please search for your issue first before filing. Links to search and create new issues:

Migration resources

Links to the repo folders for Module 20 (START) and Module 21 (FINISH) can be found in the table below.

Codelab

Python 2

Python 3

Module 20

code

(n/a)

Module 21 (this codelab)

code

code

Online references

Below are resources relevant for this tutorial:

Cloud Identity Platform and Cloud Marketplace

Cloud Resource Manager, Cloud IAM, Firebase Admin SDK

App Engine Users, App Engine NDB, Cloud NDB, Cloud Datastore

Other Migration Module references

App Engine migration

App Engine platform

Cloud SDK

Other Cloud information

Videos

License

This work is licensed under a Creative Commons Attribution 2.0 Generic License.