Module 4: Migrate from Google App Engine to Cloud Run with Docker

1. Overview

This series of codelabs (self-paced, hands-on tutorials) aims to help Google App Engine (Standard) developers modernize their apps by guiding them through a series of migrations. Once that's accomplished, users can make their apps more portable by explicitly containerizing them for Cloud Run, Google Cloud's container-hosting sister service to App Engine, and other container-hosting services.

This tutorial teaches you how to containerize App Engine apps for deploying to the Cloud Run fully-managed service using Docker, a well-known platform in industry for developing, shipping, and running applications in containers. For Python 2 developers, this tutorial STARTs with the Module 2 Cloud NDB App Engine sample app while Python 3 developers START with the Module 3 Cloud Datastore sample.

You'll learn how to

  • Containerize your app using Docker
  • Deploy container images to Cloud Run

What you'll need

Survey

How will you use this codelab?

Only read through it Read it and complete the exercises

2. Background

PaaS systems like App Engine and Cloud Functions provide many conveniences for your team and application. For example, these serverless platforms enable SysAdmin/Devops to focus on building solutions. Your app can autoscale up as needed, scale down to zero with pay-per-use billing help control costs, and they use a variety of common development languages.

However, the flexibility of containers is compelling as well, the ability to choose any language, any library, any binary. Giving users the best of both worlds, the convenience of serverless along with the flexibility of containers, is what Google Cloud Run is all about.

Learning how to use Cloud Run is not within the scope of this codelab; that's covered by the Cloud Run documentation. The goal here is for you to know how to containerize your App Engine app for Cloud Run (or other services). There are a few things you should know before moving forward, primarily that your user experience will be slightly different, a bit lower-level as you will no longer be taking application code and deploying it.

Instead, you'll need to learn something about containers like how to build and deploy them. You also get to decide what you want to put in the container image, including a web server as you won't be using App Engine's web server any more. If your prefer not to follow this path, keeping your apps on App Engine is not a bad option.

In this tutorial, you'll learn how to containerize your app, replacing App Engine config files with container configuration, determine what goes into the container, then specify how to start your app — many of these things are automatically handled by App Engine.

This migration features these steps:

  1. Setup/Prework
  2. Containerize application
    • Replace configuration files
    • Modify application files

3. Setup/Prework

Before we get going with the main part of the tutorial, let's set up our project, get the code, then deploy the baseline app so we know we started with working code.

1. Setup project

If you completed either the Module 2 or Module 3 codelabs, we recommend reusing that same project (and code). Alternatively, you can create a brand new project or reuse another existing project. Ensure the project has an active billing account and App Engine (app) is enabled.

2. Get baseline sample app

One of the prerequisites to this codelab is to have a working Module 2 or Module 3 sample app. If you don't have one, go complete either tutorial (links above) before moving ahead here. Otherwise if you're already familiar with its contents, you can just start by grabbing the Module 2 or 3 code below.

Whether you use yours or ours, the Module 2 code is where we'll START for the Python 2 version of this tutorial, and similarly, the Module 3 code for Python 3. This Module 4 codelab walks you through each step, and depending on your options, you should end up with code that resembles one of the Module 4 repo folders (FINISH) when complete.

The directory of Python 2 Module 2 STARTing files (yours or ours) should look like this:

$ ls
README.md               appengine_config.py     requirements.txt
app.yaml                main.py                 templates

If you're using your own Module 2 (2.x) code, you'll also have a lib folder. Neither lib nor appengine_config.py are used for Python 3, where the Module 3 (3.x) STARTing code should look like this:

$ ls
README.md               main.py                 templates
app.yaml                requirements.txt

3. (Re)Deploy baseline app

Your remaining prework steps to execute now:

  1. Re-familiarize yourself with the gcloud command-line tool
  2. Re-deploy the sample app with gcloud app deploy
  3. Confirm the app runs on App Engine without issue

Once you've successfully executed those steps, you're ready to containerize it.

4. Containerize application

Docker is the standard containerization platform in industry today. One challenge in using it, as mentioned earlier, is that it requires effort to curate an efficient Dockerfile, the configuration file that determines how your container images are built. Buildpacks, on the other hand, require little effort since it uses introspection to determine your app's dependencies, making the Buildpacks container as efficient as possible for your app.

You're in the right place if you already know about containers, Docker, and want to learn more about containerizing your App Engine app for Cloud Run. Feel free to also do the Module 5 codelab (identical to this one but with Cloud Buildpacks) afterwards. Our barebones sample app is lightweight enough to avoid some of those aforementioned Dockerfile issues.

The migration steps include replacing the App Engine configuration files and specifying how your app should start. Below is a table summarizing the configuration files to expect for each platform type. Compare the App Engine column with the Docker column (and optionally Buildpacks):

Description

App Engine

Docker

Buildpacks

General config

app.yaml

Dockerfile

(service.yaml)

3rd-party libraries

requirements.txt

requirements.txt

requirements.txt

3rd-party config

app.yaml (plus appengine_config.py and lib [2.x-only])

(n/a)

(n/a)

Startup

(n/a) or app.yaml (if entrypoint used)

Dockerfile

Procfile

Ignore files

.gcloudignore and .gitignore

.gcloudignore, .gitignore, and .dockerignore

.gcloudignore and .gitignore

Once your app is containerized, it can be deployed to Cloud Run. Other Google Cloud container platform options include Compute Engine, GKE, and Anthos.

General config

Migrating from App Engine means replacing app.yaml with a Dockerfile that outlines how to build and run the container. App Engine automatically starts your application, but Cloud Run doesn't. This is what the Dockerfile ENTRYPOINT and CMD commands are for. Learn more about Dockerfile from this Cloud Run docs page as well as see an example Dockerfile that spawns gunicorn.

An alternative to using ENTRYPOINT or CMD in a Dockerfile is to use a Procfile. Finally, a .dockerignore helps filter out non-app files to keep your container size down. More on these coming up!

Delete app.yaml and create Dockerfile

The app.yaml is not used in containers so delete it now. The container configuration file is Dockerfile, and our sample app only requires a minimal one. Create your Dockerfile with this content, replacing NNN with 2 or 3, depending on which Python version you're using:

FROM python:NNN-slim
WORKDIR /app
COPY . .
RUN pip install -r requirements.txt
ENTRYPOINT ["python", "main.py"]

Most of the Dockerfile specifies how to create the container except ENTRYPOINT which specifies how to start the container, in this case calling python main.py to execute the Flask development server. If you're new to Docker, the FROM directive indicates the base image to start from, and "slim" refers to a minimal Python distribution. Learn more from the Docker Python images page.

The middle set of commands creates the working directory (/app), copies in the application files, then runs pip install to bring 3rd-party libraries into the container. WORKDIR combines the Linux mkdir and cd commands together; read more about it in the WORKDIR documentation . The COPY and RUN directives are self-explanatory.

3rd-party libraries

The requirements.txt file can stay the same; Flask should be there along with your Datastore client library (Cloud Datastore or Cloud NDB). If you wish to use another WSGI-compliant HTTP server like Gunicorn — current version at the time of this writing is 20.0.4 — then add gunicorn==20.0.4 to requirements.txt.

3rd-party config

Python 2 App Engine developers know that 3rd-party libraries are either copied into the lib folder, referenced in requirements.txt, itemized in app.yaml, and supported by appengine_config.py. Containers, like Python 3 App Engine apps, only use requirements.txt, so all the other stuff can be dropped, meaning if you have a 2.x App Engine app, delete appengine_config.py and any lib folder now.

Startup

Python 2 users do not startup App Engine's web server, but when moving to a container, you do have to do this. This is done by adding a CMD or ENTRYPOINT directive in your Dockerfile specifying how to start your app, and this is described below in the same manner as for Python 3 users.

Python 3 users have the option of converting their app.yaml files to have an entrypoint instead of script: auto directives in their handlers section. If you use entrypoint in your Python 3 app.yaml, it would look something like this:

runtime: python38
entrypoint: python main.py

The entrypoint directive tells App Engine how to start your server. You can move it almost directly into your Dockerfile (or Procfile if using Buildpacks [see Module 5] to containerize your app). Summarizing where an entrypoint directive would go between both platforms:

  • Docker: line in Dockerfile: ENTRYPOINT ["python", "main.py"]
  • Buildpacks: line in Procfile: web: python main.py

Using the Flask development server is fine for testing, but if using a production server like gunicorn for your application, be sure to point your ENTRYPOINT or CMD directive at it like in the Cloud Run Quickstart sample.

Ignore files

We recommend creating a .dockerignore file to trim the size of your container and not clutter your container image with superfluous files like these:

*.md
*.pyc
*.pyo
.git/
.gitignore
__pycache__

Application files

All Module 2 or Module 3 apps are fully Python 2-3 compatible, meaning there are no changes to the core components of main.py; we will only be adding a few lines of startup code. Add a pair of lines at the bottom of main.py to start the development server as Cloud Run requires port 8080 to be open so it can call your app:

if __name__ == '__main__':
    import os
    app.run(debug=True, threaded=True, host='0.0.0.0',
            port=int(os.environ.get('PORT', 8080)))

5. Build and deploy

With the Docker configuration and source file updates completed, you're ready to get it running on Cloud Run. Before that, let's briefly discuss services.

Services vs. Apps

While App Engine was primarily created to host applications, it is also a platform for hosting web services or applications made up of a collection of microservices. In Cloud Run, everything is a service, whether it's an actual service or an application with a web interface, so consider its use as the deployment of a service rather than an appplication.

Unless your App Engine app was made up of multiple services, you really didn't have to do any kind of naming when deploying your applications. That changes with Cloud Run where you do need to come up with a service name. Whereas an App Engine's appspot.com domain features its project ID, e.g., https://PROJECT_ID.appspot.com, and perhaps it's region ID abbreviation, e.g., http://PROJECT_ID.REGION_ID.r.appspot.com.

However, the domain for a Cloud Run service features its service name, region ID abbreviation, and a hash, but not its project ID, e.g., https://SVC_NAME-HASH-REG_ABBR.a.run.app. Bottom line, start thinking of a service name!

Deploy service

Execute the command below to build your container image and deploy to Cloud Run. When prompted, select your region and allow unauthenticated connections for easier testing and select your region as appropriate where SVC_NAME is the name the service you're deploying.

$ gcloud beta run deploy SVC_NAME --source .
Please choose a target platform:
 [1] Cloud Run (fully managed)
 [2] Cloud Run for Anthos deployed on Google Cloud
 [3] Cloud Run for Anthos deployed on VMware
 [4] cancel
Please enter your numeric choice:  1

To specify the platform yourself, pass `--platform managed`. Or, to make this the default target platform, run `gcloud config set run/platform managed`.

Please specify a region:
 [1] asia-east1
 [2] asia-east2
 [3] asia-northeast1
 [4] asia-northeast2
 [5] asia-northeast3
 [6] asia-south1
 [7] asia-southeast1
 [8] asia-southeast2
 [9] australia-southeast1
 [10] europe-north1
 [11] europe-west1
 [12] europe-west2
 [13] europe-west3
 [14] europe-west4
 [15] europe-west6
 [16] northamerica-northeast1
 [17] southamerica-east1
 [18] us-central1
 [19] us-east1
 [20] us-east4
 [21] us-west1
 [22] us-west2
 [23] us-west3
 [24] us-west4
 [25] cancel
Please enter your numeric choice: <select your numeric region choice>

To make this the default region, run `gcloud config set run/region REGION`.

Allow unauthenticated invocations to [SVC_NAME] (y/N)?  y

Building using Dockerfile and deploying container to Cloud Run service [SVC_NAME] in project [PROJECT_ID] region [REGION]
✓ Building and deploying new service... Done.
  ✓ Uploading sources...
  ✓ Building Container... Logs are available at [https://console.cloud.google.com/cloud-build/builds/BUILD-HASH?project=PROJECT_NUM].
  ✓ Creating Revision... Deploying Revision.
  ✓ Routing traffic...
  ✓ Setting IAM Policy...
Done.
Service [SVC_NAME] revision [SVC_NAME-00001-vos] has been deployed and is serving 100 percent of traffic.
Service URL: https://SVC_NAME-HASH-REG_ABBR.a.run.app

Visit the specified URL with your browser to confirm the deployment was successful!

As the gcloud command indicates, users can set various default settings to reduce the output and interactivity as shown above. For example, to avoid all interaction, you can use the following one-liner deploy command instead:

$ gcloud beta run deploy SVC_NAME --source . --platform managed --region REGION --allow-unauthenticated

If you use this one, be sure to select the same service name SVC_NAME and the desired REGION name, not the indexed menu selection as we did interactively above.

6. Summary/Cleanup

Confirm that the app works on Cloud Run as it did on App Engine. If you jumped into this series without doing any of the preceding codelabs, the app itself doesn't change; it registers all visits to the main web page (/) and looks like this once you've visited the site enough times:

visitme app

Your code should now match what's in the Module 4 repo folder, whether it's 2.x or 3.x. Congrats on completing this Module 4 codelab.

Optional: Clean up

What about cleaning up to avoid being billed until you're ready to move onto the next migration codelab? Since you're now using a different product, be sure to review the Cloud Run pricing guide.

Optional: Disable service

If you're not ready to go to the next tutorial yet, disable your service to avoid incurring additional charges. When you're ready to move onto the next codelab, you can re-enable it. While your app is disabled, it won't get any traffic to incur charges, however another thing you can get billed for is your Datastore usage if it exceeds the free quota, so delete enough to fall under that limit.

On the other hand, if you're not going to continue with migrations and want to delete everything completely, you can either delete your service or shutdown your project entirely.

Next steps

Congratulations, you've containerized your app, which concludes this tutorial! From here, the next step is to learn about how to do the same thing with Cloud Buildpacks in the Module 5 codelab (link below) or work through another App Engine migration:

  • Migrate to Python 3 if you haven't already. The sample app is already 2.x and 3.x compatible, so the only change is for Docker users to update their Dockerfile to use a Python 3 image.
  • Module 5: Migrate to Cloud Run with Cloud Buildpacks
    • Containerize your app to run on Cloud Run with Cloud Buildpacks
    • You do not need to know anything about Docker, containers, or Dockerfiles
    • Requires you to have already migrated your app to Python 3
  • Module 7: App Engine Push Task Queues (required if you use [push] Task Queues)
    • Adds App Engine taskqueue push tasks to Module 1 app
    • Prepares users for migrating to Cloud Tasks in Module 8
  • Module 3:
    • Modernize Datastore access from Cloud NDB to Cloud Datastore
    • This is the library used for Python 3 App Engine apps and non-App Engine apps
  • Module 6: Migrate to Cloud Firestore
    • Migrate to Cloud Firestore to access Firebase features
    • While Cloud Firestore supports Python 2, this codelab is available only in Python 3.

7. Additional resources

App Engine migration module codelabs issues/feedback

If you find any issues with this codelab, please search for your issue first before filing. Links to search and create new issues:

Migration resources

Links to the repo folders for Module 2 and 3 (START) and Module 4 (FINISH) can be found in the table below. They can also be accessed from the repo for all App Engine codelab migrations which you can clone or download a ZIP file.

Codelab

Python 2

Python 3

Module 2

code

(code)

Module 3

(code)

code

Module 4

code

code

App Engine and Cloud Run resources

Below are additional resources regarding this specific migration: