Module 1: Migrate from App Engine webapp2 to Flask

1. Overview

This series of codelabs (self-paced, hands-on tutorials) aims to help Google App Engine (Standard) developers modernize their apps by guiding them through a series of migrations. The most significant step is to move away from original runtime bundled services because the next generation runtimes are more flexible, giving users a greater variety of service options. Moving to the newer generation runtime enables you to integrate with Google Cloud products more easily, use a wider range of supported services, and support current language releases.

This initial tutorial shows the first migration steps for modernizing the web framework in App Engine apps: moving from webapp2 to Flask. In your app, you can use any web framework that handles routing, but for this tutorial, we use Flask as it's broadly used by the community.

You'll learn how to

  • Use third-party libraries (built-in or otherwise)
  • Update configuration files
  • Migrate a simple app from webapp2 to Flask

What you'll need

Survey

How will you use this codelab?

Only read through it Read it and complete the exercises

2. Background

The webapp framework was bundled when App Engine first launched on Python 2.5 in 2008. Years later it was replaced by successor webapp2 when the 2.7 runtime deprecated 2.5 in 2013.

While webapp2 (see docs) still exists and can be used outside of App Engine as a WSGI-compliant web framework, it does not do its own routing of user requests to the appropriate code in the application. Instead, it relies on App Engine, configuration files, and the developer to perform that routing of web traffic to corresponding "handlers." Furthermore, webapp2's core benefits are inextricably tied to App Engine's bundled services, effectively deprecating it even though it works on Python 3 (also see related issue).

This module gives practioners hands-on experience for migrating a simple webapp2 app to Flask, a framework supported by App Engine and many more services outside of Google Cloud, making apps much more portable. If Flask is not a desired framework to move your application to, you may select another as long as it does its own routing. This codelab shows information technology decision-makers (ITDMs) and developers what the migration steps are, so you can familiarize yourself with this process regardless of which framework you actually migrate to.

These are the primary steps for this migration:

  1. Setup/Prework
  2. Add Flask 3rd-party library
  3. Update application files
  4. Update HTML template file

3. Setup/Prework

Before we get going with the main part of the tutorial, let's setup our project, get the code, then (re)familiarize you with the gcloud command and deploy the baseline app so we know we started with working code.

1. Setup project

As an existing developer, your App Engine dashboard likely already shows what services you've got running. For the purposes of this tutorial, we recommend you create a brand new project or reuse an existing one for this tutorial. Ensure the project has an active billing account and App Engine (app) is enabled.

2. Download baseline sample app

The GAE migration repo has all the code you need. Clone it or download its ZIP file. For this tutorial, you'll start with the code in Module 0 folder (START), and when you've completed the tutorial, your code should match the Module 1 folder (FINISH). If not, check out the differences so you can move onto the next lab.

The Module 0 folder should have files that look like this, as illustrated with the POSIX ls command:

$ ls
app.yaml        index.html      main.py

3. (Re)Familiarize yourself w/gcloud commands

If you don't have the gcloud command on your machine yet, install the Google Cloud SDK and ensure gcloud is available as part of your execution path and familiarize yourself with the following gcloud commands:

  1. gcloud components update — update Google Cloud SDK
  2. gcloud auth login — login to your credentialed account
  3. gcloud config list — list GCP project configuration settings
  4. gcloud config set project PROJECT_ID — set GCP project ID
  5. gcloud app deploy — deploy your App Engine application

If you haven't been doing App Engine development with gcloud recently, you should run the first four commands (#1-#4) to get set up before moving to the next steps. Let's do a quick overview of these commands.

Firstly, gcloud components update ensures you have the latest Cloud SDK version. Running this command should give an output like the following:

$ gcloud components update

Your current Cloud SDK version is: 317.0.0
You will be upgraded to version: 318.0.0

┌──────────────────────────────────────────────────┐
│        These components will be updated.         │
├──────────────────────────┬────────────┬──────────┤
│           Name           │  Version   │   Size   │
├──────────────────────────┼────────────┼──────────┤
│ Cloud SDK Core Libraries │ 2020.11.06 │ 15.5 MiB │
│ gcloud cli dependencies  │ 2020.11.06 │ 10.6 MiB │
└──────────────────────────┴────────────┴──────────┘

The following release notes are new in this upgrade.
Please read carefully for information about new features, breaking changes,
and bugs fixed.  The latest full release notes can be viewed at:
  https://cloud.google.com/sdk/release_notes

318.0.0 (2020-11-10)

      . . .
      (release notes)
      . . .

    Subscribe to these release notes at
    https://groups.google.com/forum/#!forum/google-cloud-sdk-announce.

Do you want to continue (Y/n)?

╔════════════════════════════════════════════════════════════╗
╠═ Creating update staging area                             ═╣
╠════════════════════════════════════════════════════════════╣
╠═ Uninstalling: Cloud SDK Core Libraries                   ═╣
╠════════════════════════════════════════════════════════════╣
╠═ Uninstalling: gcloud cli dependencies                    ═╣
╠════════════════════════════════════════════════════════════╣
╠═ Installing: Cloud SDK Core Libraries                     ═╣
╠════════════════════════════════════════════════════════════╣
╠═ Installing: gcloud cli dependencies                      ═╣
╠════════════════════════════════════════════════════════════╣
╠═ Creating backup and activating new installation          ═╣
╚════════════════════════════════════════════════════════════╝

Performing post processing steps...done.

Update done!

To revert your SDK to the previously installed version, you may run:
  $ gcloud components update --version 317.0.0

Next, use gcloud auth login to authenticate yourself for gcloud commands you'll issue moving forward:

$ gcloud auth login
Your browser has been opened to visit:

    https://accounts.google.com/o/oauth2/auth?response_type=code&client_id= . . .

You are now logged in as [YOUR_EMAIL].
Your current project is [PROJECT_ID].  You can change this setting by running:
  $ gcloud config set project PROJECT_ID

Use gcloud config list to see what your current project settings are:

$ gcloud config list
[core]
account = YOUR_EMAIL
disable_usage_reporting = False
project = PROJECT_ID

Your active configuration is: [default]

The command above should guide you in either creating a new project or selecting an existing one. If the output of gcloud config list doesn't match the selected project you intend to use for this tutorial, run gcloud config set project PROJECT_ID to set the project ID. Then, confirm the right project ID is set by running gcloud config list again.

$ gcloud config set project PROJECT_ID
Updated property [core/project].

If you prefer to use the Cloud Console instead, you can follow the user interface to create a new project if desired or use any pre-existing project you already have. On your project's dashboard, you should see the project info card which shows its ID (along with the project name and number):

project info card

The last command (#5), gcloud app deploy, is for deploying your app to App Engine. Since we're just starting, running it now is optional, but we certainly don't discourage deploying the Module 0 code to confirm it works. Upon execution, select the geographic region you want the app to run in (typically where you're located). It can't be changed once set however. Then watch the rest of the deployment information. When it has completed, you'll be notified of the URL your app will be served at. Here's an abridged version of what you might see:

$ gcloud app deploy
Services to deploy:

descriptor:      [/private/tmp/mod0-baseline/app.yaml]
source:          [/private/tmp/mod0-baseline]
target project:  [PROJECT_ID]
target service:  [default]
target version:  [20201116t220827]
target url:      [https://PROJECT_ID.REG_ABBR.r.appspot.com]


Do you want to continue (Y/n)?

Beginning deployment of service [default]...
╔════════════════════════════════════════════════════════════╗
╠═ Uploading 1 file to Google Cloud Storage                 ═╣
╚════════════════════════════════════════════════════════════╝
File upload done.
Updating service [default]...done.
Setting traffic split for service [default]...done.
Deployed service [default] to [https://PROJECT_ID.REG_ABBR.r.appspot.com]

You can stream logs from the command line by running:
  $ gcloud app logs tail -s default

To view your application in the web browser run:
  $ gcloud app browse

If you haven't used App Engine in a while, you might notice that the original deployment appcfg.py update command has been replaced by gcloud app deploy. To learn more about gcloud app deploy, check out its documentation page.

Another recent change is the deployed apps' URL, tweaked from http://PROJECT_ID.appspot.com to http://PROJECT_ID.REG_ABBR.r.appspot.com. Most apps will eventually be converted to the new format. More info on the URL format in the requests and routing documentation.

After the app is deployed, refresh the browser (possibly a few times) to see the latest visits:

visitme app

If your app is new, you'll see only one or a few visits.

4. Add Flask 3rd-party library

The Python 2 App Engine runtime provides a set of "built-in" third-party libraries where all you need to do is specify them in your app.yaml file to use. While this migration doesn't require their use, they will be in the next migration tutorial (for Module 2).

Third-party libraries which are not built-in must be specified in a file called requirements.txt and installed locally in the lib folder in the same directory as the application code where everything is uploaded to App Engine. The documentation for bundling 3rd-party libraries has more information.

Copied libraries like Flask require that you tell App Engine to look for them in the lib folder using the appengine_config.py configuration file. The appengine_config.py configuration file is placed in the same top-level application folder as requirements.txt and lib. In this part of the tutorial, you will:

  • Create requirements.txt (specify copied [non-built-in] 3rd-party libraries)
  • Create appengine_config.py (recognize 3rd-party libraries)
  • Install (3rd-party) packages and dependencies

1. Create requirements.txt

Create a requirements.txt file to specify your packages. In our case, Flask is the third-party library that's needed. At the time of this writing, the latest version is 1.1.2, so create requirements.txt with this one line:

Flask==1.1.2

Refer to the requirements.txt documentation to learn more about accepted formats.

2. Create appengine_config.py

The next step is to have App Engine recognize external 3rd-party libraries. Create a file named appengine_config.py with the following contents:

from google.appengine.ext import vendor

# Set PATH to your libraries folder.
PATH = 'lib'
# Add libraries installed in the PATH folder.
vendor.add(PATH)

This code does exactly what we specified earlier, that is, point App Engine to the lib folder for copied libraries.

3. Install package(s) and dependencies

Now run the pip install command to create the lib folder and install Flask and its dependencies there:

$ pip install -t lib -r requirements.txt

Whether you used pip or pip2, after the package installation has completed, you should have a lib folder that will have contents similar to:

$ ls lib
bin/
click/
click-7.1.2.dist-info/
flask/
Flask-1.1.2.dist-info/
itsdangerous/
itsdangerous-1.1.0.dist-info/
jinja2/
Jinja2-2.11.2.dist-info/
markupsafe/
MarkupSafe-1.1.1.dist-info/
werkzeug/
Werkzeug-1.0.1.dist-info/

5. Update application files

Now let's update the application file, main.py.

1. Imports

The imports come first as in all Python files. The webapp2 framework import is followed by the ndb Datastore library, and finally, the App Engine extension that processes Django-flavored templates. You should see the following:

  • BEFORE:
import webapp2
from google.appengine.ext import ndb
from google.appengine.ext.webapp import template

When moving to Flask, you import both Flask and the template renderer pieces at the same time. Delete the pair of webapp2-related imports and replace them as follows (leave the ndb import as-is):

  • AFTER:
from flask import Flask, render_template, request
from google.appengine.ext import ndb

2. Startup

Apps using webapp2 require a single array (Python list) that lists all the routes and handlers in any Python file (there could be others):

  • BEFORE:
app = webapp2.WSGIApplication([
    ('/', MainHandler),
], debug=True)

Bear in mind that app.yaml performs higher-level routing and may call different handlers. The sample app is simple enough that all routes come to the main.py handler.

Flask doesn't use routing tables like this, so delete these lines in main.py. Flask also requires initialization, so add the following line at the top of main.py just under the imports:

  • AFTER:
app = Flask(__name__)

In Flask, you initialize the framework then use decorators to define the routes. Also, routes are paired with functions, not classes nor methods.

It is out-of-scope to include a Flask tutorial in this codelab, so spend some time working through the Flask tutorial and review the Flask documentation to get more comfortable with the framework.

3. Data model

There are no changes here. Datastore will be the focus of the next codelab.

4. Handlers

The application, regardless which framework you use (webapp2 or Flask), does 3 things:

  1. Handle root path (/) GET requests
  2. Register a web page "visit" (create/store Visit object)
  3. Display the top 10 most recent visits (with a pre-defined template, index.html)

The webapp2 framework uses a class-based execution model where handlers are created for each supported HTTP method. In our simple case, we only have GET, so a get() method is defined:

  • BEFORE:
class MainHandler(webapp2.RequestHandler):
    def get(self):
        store_visit(self.request.remote_addr, self.request.user_agent)
        visits = fetch_visits(10) or ()  # empty sequence if None
        tmpl = os.path.join(os.path.dirname(__file__), 'index.html')
        self.response.out.write(template.render(tmpl, {'visits': visits}))

As mentioned above, Flask does its own routing. Instead of a handler class, you write functions and decorate them with the route they should be called for. Users can specify HTTP methods handled in the decorator call, i.e., @app.route('/app/', methods=['GET', 'POST']). Since the default is only GET (and implicitly HEAD), it can be left off.

In migrating to Flask, replace the MainHandler class and its get() method with the following Flask routing function:

  • AFTER:
@app.route('/')
def root():
    store_visit(request.remote_addr, request.user_agent)
    visits = fetch_visits(10) or ()  # empty sequence if None
    return render_template('index.html', visits=visits)

Of course this isn't representative of your app which will certainly be more complex than this sample. One primary goal for these tutorials is to help you get started, build some of that "muscle memory," and understand where to make changes in App Engine-specific code. To confirm you've made this change correctly, compare yours to the Module 1 main.py.

5. Auxiliary files

There are no changes to the .gcloudignore file. Its purpose is to specify files to not deploy to App Engine that are unnecessary to deploy and execute the application, including but not limited to auxiliary Python, source control, repo boilerplate, and other files. Our .gcloudignore looks like this (with comments removed for brevity):

.gcloudignore
.git
.gitignore
.hgignore
.hg/
*.pyc
*.pyo
__pycache__/
/setup.cfg
README.md

6. Update HTML template file

1. Move template file

In the baseline repo folder (Module 0), the index.html template file is in the same folder as the application files. Since Flask requires HTML files placed in a templates folder, you must create that folder (mkdir templates) and move index.html in there. In a POSIX-compliant system like Linux or Mac OS X, the commands would be:

mkdir templates
mv index.html templates

2. Update template file

Once you've moved index.html into templates, it is time to make a small but required edit. Let's take a look at the original template file in its entirety:

<!doctype html>
<html>
<head>
<title>VisitMe Example</title>
<body>

<h1>VisitMe example</h1>
<h3>Last 10 visits</h3>
<ul>
{% for visit in visits %}
    <li>{{ visit.timestamp.ctime }} from {{ visit.visitor }}</li>
{% endfor %}
</ul>

</body>
</html>

Whereas webapp2 uses Django templates which execute callables like visit.timestamp.ctime without parentheses ( ), Jinja2 requires them explicitly. While this sounds like a minor tweak, Jinja templates are more powerful out-of-the-box because you can pass arguments in calls.

In Django, you either have to create a "template tag" or write a filter. With this understanding, update index.html by adding a pair of parentheses to the visit.timestamp.ctime call:

  • BEFORE:
<li>{{ visit.timestamp.ctime }} from {{ visit.visitor }}</li>
  • AFTER:
<li>{{ visit.timestamp.ctime() }} from {{ visit.visitor }}</li>

This is the only change required; no additional changes to index.html are required for all remaining migration codelabs.

7. Summary/Cleanup

Deploy application

When you've completed all the changes in this tutorial, the files in your application folder should be identical (or nearly so) to the file in the Module 1 repo folder. Now deploy and see that your Module 1 Flask application runs identically to the Module 0 webapp2 version.

Use the gcloud app deploy command as we did earlier when deploying the original Module 0 code. Accessing the app at PROJECT_ID.appspot.com, whether from a web browser or a curl or wget command to confirm it works as expected.

If you get some kind of server error, that typically means some sort of typo in your Python code. Take a look at your application logs to investigate. Also compare your files with those in the Module 1 repo (link just above).

Optional: Clean up

What about cleaning up to avoid being billed until you're ready to move onto the next migration codelab? As existing developers, you're likely already up-to-speed on App Engine's pricing information.

Optional: Disable app

If you're not ready to go to the next tutorial yet, disable your app to avoid incurring charges. When you're ready to move onto the next codelab, you can re-enable it. While your app is disabled, it won't get any traffic to incur charges, however another thing you can get billed for is your Datastore usage if it exceeds the free quota, so delete enough to fall under that limit.

On the other hand, if you're not going to continue with migrations and want to delete everything completely, you can shutdown your project.

Next steps

There are two migration modules that START with completed Module 1 code, Modules 2 and 7:

  • Module 2 (required if you use Datastore)
    • Migrate from App Engine ndb to Cloud NDB
    • After switching to Cloud NDB, many other options become available
      • Containerizing your app to run on Cloud Run
      • Further migrating your app to Cloud Datastore client library
      • Migrating your app to Cloud Firestore to access Firebase features
  • Module 7 (required if you use [push] Task Queues)
    • Add App Engine (push) taskqueue usage
    • Prepares Module 1 app for migration to Cloud Tasks in Module 8

8. Additional resources

App Engine migration module codelabs issues/feedback

If you find any issues with this codelab, please search for your issue first before filing. Links to search and create new issues:

Migration resources

Links to the repo folders for Module 0 (START) and Module 1 (FINISH) can be found in the table below. They can also be accessed from the repo for all App Engine migrations which you can clone or download a ZIP file.

Codelab

Python 2

Python 3

Module 0

code

(n/a)

Module 1

code

(n/a)

App Engine resources

Below are additional resources regarding this specific migration: