Step 1: Migrate from webapp2 to Flask

This series of tutorials is designed to aid Google App Engine (Standard) developers modernize their apps by migrating to the next generation runtimes. The first step is to move away from App Engine's original bundled services. The first step for Python developers is to move from the bundled webapp2 web framework to Flask, allowing apps to run outside App Engine. Another reason for this migration is that App Engine now requires frameworks that do their own routing.

What you'll learn

  • Using of external third-party libraries
  • Updating configuration files
  • Migrating a simple web app from webapp2 to Flask

What you'll need

Survey

How will you use this codelab?

Only read through it Read it and complete the exercises

App Engine's original runtimes featured bundled services that are no longer available for next generation runtimes. The webapp framework was bundled when App Engine first launched on Python 2.5 in 2008. Years later it was replaced by successor webapp2 when the 2.7 runtime deprecated 2.5 in 2013.

While webapp2 (see docs) still exists and can be used outside of App Engine as a WSGI-compliant web framework, it doesn't have perform routing, and its core benefits are inextricably tied App Engine's bundled services, effectively deprecating it even though it works on Python 3 (also see related issue).

The purpose of this step is to show you a simple webapp2 app and how you would migrate it to Flask, a web framework supported by App Engine and many more services outside of Google Cloud, making apps much more portable. Even if you do not have a web UI, you still need to migrate your app because `webapp2 is not supported in the next-gen runtimes. You must select a web framework that supports routing. This purpose of this codelab is to show ITDMs & developers what the migration steps are.

These are the primary steps for this migration:

  1. Setup/Prework
  2. Add Flask 3rd-party library
  3. Update application files

There are 3 objectives in this part of the codelab:

  1. Setup project/application
  2. Download baseline sample app
  3. (Re)Familiarize yourself w/gcloud commands

As an existing developer, your App Engine dashboard likely already shows what services you've got running. For the purposes of this tutorial, we recommend you create a brand new project (and application), or reuse an existing one that's been domant, to experiment with migration.

Grab all the code for the starter app in this codelab's repo (clone it or download a ZIP file). Specifically, we're going to start with the starter app in repo Step 0 and slowly work on it in this tutorial until it turns into repo Step 1, the state your app s/b in when this codelab concludes.

We're going to convert that to web app to Flask. When you've downloaded the files, you should have these in your working directory:

$ ls
app.yaml        index.html      main.py

You may be familiar with the appcfg.py update command to deploy your app. That has now been replaced by gcloud app deploy. If you don't have the gcloud command on your machine yet, install the Google Cloud SDK to get it and ensure gcloud is available as part of your execution path. Specifically, you should be(come) familiar with the following gcloud commands:

  • gcloud auth login — login to your credentialed account
  • gcloud config set project PROJECT_ID — set your GCP project
  • gcloud app deploy — deploy your App Engine application

If you haven't been doing App Engine development with gcloud recently, you'll likely have to run all 3 of these commands. Before moving on, ensure you can deploy the sample app with the last command above. Then open a browser tab to the application to confirm it deployed successfully. Moving on doesn't make sense if you can't deploy the Step 0 sample app. It should look something like this in your browser:

visitme app

The Python 2 App Engine runtime provides a set of "built-in" or "bundled" third-party libraries where all you need to do is specify them in your app.yaml file to use. Those that are not must be specified in a packages file called requirements.txt and installed locally in the lib folder in the same directory as the application code so everything is uploaded to App Engine. This process is also known as "vendoring."

Flask falls into this category (not built-in) and must be vendored-in. Vendored libraries also require you to tell App Engine to look for them in the lib folder, and that's what the appengine_config.py configuration file is for. It should be placed in the same top-level application folder as requirements.txt and lib.

Summarizing the steps to migrate web frameworks:

  1. Create requirements.txt (specify 3rd-party libraries)
  2. Create appengine_config.py (recognize 3rd-party libraries)
  3. Install 3rd-party packages & dependencies

Create requirements.txt

This step is to add Flask as a third-party library. Create a requirements.txt file that should have your packages listed. In our case, Flask. At the time of this writing, the latest version is 1.1.2, so here is the one in the repo:

Flask==1.1.2

You can also set a boundary like this, specifying 1.1.2 or newer:

Flask>=1.1.2

Also check the requirements.txt documentation for more information on accepted formats.

Create appengine_config.py

The next step is to have App Engine recognize external 3rd-party ("vendor") libraries. Create a file named appengine_config.py with the following contents:

from google.appengine.ext import vendor

# Set PATH to your libraries folder.
PATH = 'lib'
# Add libraries installed in the PATH folder.
vendor.add(PATH)

This code does exactly what we specified earlier, that is, point App Engine to the lib folder for vendored libraries.

Install package(s) & dependencies

Now that we're done with the configuration files, realize that lib doesn't exist yet or is empty, so use the pip install command to get Flask and its dependencies:

$ pip install -t lib -r requirements.txt

You'll see the standard package installation output. After the command completes, you should have a lib folder that will look similar to:

$ ls lib
bin/
click/
click-7.1.2.dist-info/
flask/
Flask-1.1.2.dist-info/
itsdangerous/
itsdangerous-1.1.0.dist-info/
jinja2/
Jinja2-2.11.2.dist-info/
markupsafe/
MarkupSafe-1.1.1.dist-info/
werkzeug/
Werkzeug-1.0.1.dist-info/

If you end up porting your app from Python 2 to 3, one key benefit is that in the next generation App Engine runtime no longer requires self-bundling third-party libraries; you simply list them in requirements.txt. There's no lib folder nor appengine_config.py file.

Feel free to review all the completed steps in the documentation for copying 3rd-party libraries.

Port from webapp2 to Flask

Now let's make the individual changes necessary, starting with the application file, main.py.

Imports

BEFORE:

Starting with the imports, with webapp2, you import both the framework library as well as the App Engine extension to process Django-flavored templates, so this is what is at the top of the Step 0 main.py:

import webapp2
from google.appengine.ext.webapp import template

AFTER:

Flask uses Jinja2 templates instead. They're integrated so it only takes one import statement that you use at the same time as importing Flask itself, so delete the old webapp2 imports and replace them with:

from flask import Flask, render_template, request

Startup

BEFORE:

webapp2 apps are initialized with all the routes for this handler in a single array (Python list), so this is what is currently in our main.py file:

app = webapp2.WSGIApplication([
    ('/', MainHandler),
], debug=True)

AFTER:

Now replace those lines of code in main.py with just this one:

app = Flask(__name__)

In Flask, you merely initialize the framework and use decorators to define the routes. As such, the call is much simpler. Route and function, not classes and methods. This codelab barely scratches the surface in teaching you a completely new web framework. It's highly advisable you spend time with the Flask tutorial and the rest of the Flask documentation. Doing this will familiarize you better with Flask and better prepare you for when we update the handlers below.

Data model

There's no change required for Datastore access part of the code. (Don't worry, Datastore will be the focus of the next tutorial.)

Handlers

BEFORE:

This app has only one method of execution: handle requests to root (/), register this visit, and display the top 10 most recent "visits" via a pre-defined template file (index.html). The webapp2 framework uses class-based execution model where handlers are created for each supported HTTP method. In our simple case, we only have GET, as we can see in main.py:

class MainHandler(webapp2.RequestHandler):
    def get(self):
        store_visit(self.request.remote_addr, self.request.user_agent)
        visits = fetch_visits(10) or ()  # empty sequence if None
        tmpl = os.path.join(os.path.dirname(__file__), 'index.html')
        self.response.out.write(template.render(tmpl, {'visits': visits}))

AFTER:

As mentioned above, Flask does its own routing. Instead of a handler class, you write functions and decorate them with the route they should be called for. Users can specify HTTP methods handled in the decorator call, i.e., @app.route('/app/', methods=['GET', 'POST']). Since the default is only GET (and implicitly HEAD), it can be left off. Now that you know how it works, replace the MainHandler class and its get() method with this single Flask routing function and notice that it's equivalent to MainHandler.get() but just "spelled" slightly differently:

@app.route('/')
def root():
    store_visit(request.remote_addr, request.user_agent)
    visits = fetch_visits(10) or ()  # empty sequence if None
    return render_template('index.html', visits=visits)

And that's really all you have to do for main.py. We know this isn't representative of your app which will certainly be more complex than this sample, but the point is to to help you get started, build some of that "finger" memory, and focus on changes to App Engine-specific code. To confirm you've made this change correctly, compare yours to the Step 1 main.py.

Auxiliary files

The purpose of .gcloudignore is to specify files to not deploy to App Engine. This basically drops uploading of superfluous Python, source control, and other files that should be ignored because they're not directly associated with the application. Stripping out comments, our .gcloudignore looks like this:

.gcloudignore
.git
.gitignore
.hgignore
.hg/
*.pyc
*.pyo
__pycache__/
/setup.cfg
README.md

There is no change from the Step 0 .gcloudignore.

Move template file

Flask requires HTML files placed in a templates folder, so create the folder and move index.html there. Now go into templates to make a small required edit to index.html.

Update template file

One tiny change is required in index.html... a fairly short one:

<!doctype html>
<html>
<head>
<title>VisitMe Example</title>
<body>

<h1>VisitMe example</h1>
<h3>Last 10 visits</h3>
<ul>
{% for visit in visits %}
    <li>{{ visit.timestamp.ctime }} from {{ visit.visitor }}</li>
{% endfor %}
</ul>

</body>
</html>

BEFORE:

Whereas webapp2 uses Django templates which execute callables without parentheses ( ), Jinja2 requires them explicitly. While this sounds like a minor tweak, Jinja templates are more powerful out-of-the-box because you can pass arguments in calls. (With Django, you either have to create a template tag or write a filter.) This line in index.html needs updating:

<li>{{ visit.timestamp.ctime }} from {{ visit.visitor }}</li>

AFTER:

Add a pair of parentheses to the visit.timestamp.ctime call:

<li>{{ visit.timestamp.ctime() }} from {{ visit.visitor }}</li>

This is the only change required; index.html requires no additional changes for all remaining migration tutorial steps.

When you've completed all the changes in this tutorial, the files in your application folder should be identical to the Step 1 files in the repo. If you've made any mistakes along the way, just compare your file with its equivalent in Step 1.

Step 1 files

Now deploy and see that your Step 1 Flask application runs identically to the Step 0 webapp2 version. What about cleaning up to avoid being billed until you're ready to move onto the next migration codelab? As existing developers, you're likely already up-to-speed on App Engine's pricing information.

If you're not ready to go to the next tutorial yet, disable your app to avoid incurring charges until you're ready to move on, at which point you can re-enable it. While your app is disabled, it won't get any traffic to incur charges, however another thing you can get billed for is your Datastore usage if it exceeds the free quota, so delete enough to fall under that limit. On the other hand, if you're not going to continue on and want to delete everything completely, you can shutdown your project.

The next codelab (Step 2) leads you through migrating the same sample app in this Step 1 state from the App Engine NDB to Google Cloud NDB libraries. This represents a key step because after you switch to Cloud NDB, many more doors become open to you as far as additional migrations go.

In the original App Engine runtimes, e.g., Python 2, users can have multiple handlers for their web apps, each defined in app.yaml, where those handlers could be in different application files. If you decide to port your app from Python 2 to 3, be aware that the next-generation runtime only supports web frameworks that perform their own routing, meaning all requests will be handled by a single application entrypoint (and subsequently, all handlers must be set to auto). Learn more about this in the links below:

That said, don't migrate to Python 3 yet as there is no App Engine NDB library available for 3.x. Once you complete Step 2, you would've migrated the sample app to the Cloud NDB library, and that is available in both Python 2 & 3.