Migrate from Google App Engine to Cloud Run with Cloud Buildpacks (Module 5)

The Google App Engine (GAE) migration modules show App Engine (Standard) developers how to modernize their apps running on the original legacy runtimes to the next generation platform. Once that's accomplished, they can make their apps more portable by explicitly containerizing them for Cloud Run, Google Cloud's container-hosting sister service to App Engine, and other container-hosting services.

This tutorial teaches you how to containerize App Engine apps for deploying to the Cloud Run fully-managed service using Cloud Buildpacks, an alternative to Docker which containerize your apps without managing Dockerfiles or even knowing anything about Docker. The codelab's target audience: Python 2 App Engine developers which have moved their apps away from the original built-in services and ported them to Python 3. The codelab STARTs with either a completed Module 2 or Module 3 Python 3 app.

You'll learn how to

  • Containerize your app using Cloud Buildpacks
  • Deploy container images to Cloud Run

What you'll need

Survey

How will you use this codelab?

Only read through it Read it and complete the exercises

PaaS systems like App Engine and Cloud Functions provide conveniences of serverless platforms like focus away from SysAdmin/DevOps towards building solutions, autoscaling up as needed, scaling down to 0 and pay-per-use billing to control costs, and variety of common development languages. However, the flexibility of containers is compelling as well, the ability to choose any language, any library, any binary. Giving users the best of both worlds, the convenience of serverless along with the flexibility of containers, is what Google Cloud Run is all about.

Learning how to use Cloud Run is not within the scope of this tutorial; that's covered by the Cloud Run documentation. By the end of this tutorial, you'll know how to containerize your App Engine app for Cloud Run (or other services). There are a few things you should know before moving forward, primarily that your user experience will be slightly different, a bit lower-level as you will no longer be taking application code and deploying it.

Instead, you'll need to learn something about containers like how to build and deploy them. You also get to decide what you want to put in the container image, including a web server as you won't be using App Engine's web server any more. If your prefer not to follow this path, keeping your apps on App Engine is not a bad option.

In this tutorial, you'll learn how to containerize your app, removing App Engine configuration files, manage a web server, including how to start your app.

This migration features these steps:

  1. Setup/Prework
  2. Containerize application
    • Replace configuration files
    • Modify application files

Before we get going with the main part of the tutorial, let's setup our project, get the code, then deploy the baseline app so we know we started with working code.

1. Setup project

If you completed either the Module 2 or Module 3 codelabs, we recommend reusing that same project (and code). Alternatively, you can create a brand new project or reuse another existing project. Ensure the project has an active billing account and App Engine (app) is enabled.

2. "Get" baseline sample app

One of the prerequisites to this codelab is to have a working Module 2 or 3 sample app. If you don't have one, we recommend completing either tutorial (links above) before moving ahead here. Otherwise if you're already familiar with their contents, you can just start by grabbing one of them below.

Whether you use yours or ours, that's where this tutorial STARTs. This codelab walks you through the migration, and by the time you're done, it should mostly match what's in the Module 5 FINISH repo folder.

The directory of the START files (yours or ours) should look like this:

$ ls
README.md               main.py                 templates
app.yaml                requirements.txt

3. (Re)Deploy baseline app

Your remaining prework steps to execute now:

  1. Re-familiarize yourself with the gcloud command-line tool
  2. Re-deploy sample app with gcloud app deploy
  3. Confirm app runs on App Engine without issue

Once you've successfully executed those steps, you're ready to containerize it.

Docker is the standard containerization platform in industry today. One challenge in using it, as mentioned earlier, is that it requires effort to curating an efficient Dockerfile, the configuration file that determines how your container images are built. The Buildpacks open specification lift this burden off the developer, using introspection to determine your app's dependencies, and builds the most efficient container for your app as possible.

You're in the right place if you want to containerize your App Engine app to run on Cloud Run or other container hosting platforms but without having to learn about Docker. Feel free to also do the Module 4 codelab (identical to this one but with Docker) afterwards to gain a more complete picture of the container space.

The migration steps include replacing the App Engine configuration files and specifying how your app should start. Below is a table summarizing the configuration files to expect for each platform type:

Description

App Engine

Docker

Buildpacks

General config

app.yaml

Dockerfile

service.yaml

3rd-party libraries

requirements.txt

requirements.txt

requirements.txt

3rd-party config

app.yaml (plus appengine_config.py & lib [2.x-only])

(n/a)

(n/a)

Startup

(n/a) or app.yaml (if entrypoint used)

Dockerfile

Procfile

Ignore files

.gcloudignore & .gitignore

.gcloudignore, .gitignore, & .dockerignore

.gcloudignore & .gitignore

Once your app is containerized, it can be deployed to Cloud Run, as we will do in this tutorial. Other Google Cloud container platform options include Compute Engine, GKE, and Anthos.

General config

App Engine automatically starts your application, but Cloud Run doesn't. This is what the Procfile and serves a similar role as the app.yaml entrypoint directive. Our Procfile will execute python main.py to start the Flask development server. You may also use a production web server like gunicorn if desired, and if so, be sure to add it to requirements.txt. Learn more about how to deploy from source code using Buildpacks from this Cloud Run docs page.

Since we're migrating away from App Engine, app.yaml will be replaced by an equivalent service.yaml file (automatically-generated). You only need to move your app.yaml entrypoint directive into a Procfile.

xxx TODO your app's service.yaml

3rd-party libraries

The requirements.txt file does not need to change; Flask should be there along with your Datastore client library (Cloud Datastore or Cloud NDB). If you wish to use another WSGI-compliant HTTP server like Gunicorn — current version at the time of this writing is 20.0.4 — then add gunicorn==20.0.4 to requirements.txt.

3rd-party config

In the Python 2 App Engine runtime, users specify built-in third-party libraries (those already available on Google servers) in the libraries section of app.yaml. Any non-built-in third-party libraries must be specified in requirements.txt and pip install -t lib requirements.txt called to bundle them with application code in the lib folder. Furthermore, an appengine_config.py file is required to point to the bundled packages in lib.

Specifying built-in libraries in app.yaml, bundling non-built-in libraries with pip install, and having a appengine_config.py file are all unnecessary for the Python 3 App Engine runtime. More information on these differences can found in the Python 3 migration configuration documentation.

Those files are not used with Docker nor Buildpacks either, so if you have a 2.x App Engine app, delete appengine_config.py and lib now.

Startup

Python 2 users never needed to startup App Engine's web server, but Python 3 users have the option of converting their app.yaml files to have an entrypoint instead of script: auto directives in their handlers section as described in the previous Cloud NDB migration codelab (look in the "Simplify app.yaml" section in the BONUS section on Python 3 migration [Step 7]). If you elected to convert your app.yaml to use entrypoint, it would look something like this:

runtime: python38
entrypoint: python main.py

This translates directly to the below; also showing the Docker equivalent as an FYI:

  • Buildpacks: A Procfile consisting of just this line: web: python main.py
  • Docker: This line in Dockerfile: ENTRYPOINT ["python", "main.py"]

For testing and staging, it's easy to just run Flask's development server from Python as we have above, but developers can opt for something more powerful for production such as the Cloud Run Quickstart sample which uses gunicorn.

Application files

Both the Module 2 or Module 3 apps are fully Python 2-3 compatible, meaning there are no changes to the core components of main.py; we will only be adding a few lines of startup code. Add a pair of lines at the bottom of main.py to start the development server as Cloud Run requires port 8080 to be open so it can call your app:

if __name__ == '__main__':
    import os
    app.run(debug=True, threaded=True, host='0.0.0.0',
            port=int(os.environ.get('PORT', 8080)))

With your App Engine configuration replaced by Buildpacks, you're ready to deploy. Execute the command below to build your container image and archive to the Cloud Registry.

SVC_NAME is the name the service you're deploying.

$ gcloud run deploy SVC_NAME --source .
Please choose a target platform:
 [1] Cloud Run (fully managed)
 [2] Cloud Run for Anthos deployed on Google Cloud
 [3] Cloud Run for Anthos deployed on VMware
 [4] cancel
Please enter your numeric choice:  1

To specify the platform yourself, pass `--platform managed`. Or, to make this the default target platform, run `gcloud config set run/platform managed`.

Please specify a region:
 [1] asia-east1
 [2] asia-east2
 [3] asia-northeast1
 [4] asia-northeast2
 [5] asia-northeast3
 [6] asia-south1
 [7] asia-southeast1
 [8] asia-southeast2
 [9] australia-southeast1
 [10] europe-north1
 [11] europe-west1
 [12] europe-west2
 [13] europe-west3
 [14] europe-west4
 [15] europe-west6
 [16] northamerica-northeast1
 [17] southamerica-east1
 [18] us-central1
 [19] us-east1
 [20] us-east4
 [21] us-west1
 [22] cancel
Please enter your numeric choice: REGION

To make this the default region, run `gcloud config set run/region REGION`.

Building using Buildpacks and deploying container to Cloud Run service [SVC_NAME] in project [PROJECT_ID] region [REGION]
✓ Building and deploying... Done.
  ✓ Uploading sources...
  ✓ Building Container... Logs are available at [https://console.cloud.google.com/cloud-build/
  builds/dd9cece4-0b95-4b9b-ae31-92a470c73e3d?project=1046901301037].
  ✓ Creating Revision...
  ✓ Routing traffic...
Done.
Service [SVC_NAME] revision [SVC_NAME-00014-soc] has been deployed and is serving 100 percent of traffic.
Service URL: https://SVC_NAME-HASH-REG_ABBR.a.run.app

Visit the specified URL with your browser to confirm the deployment was successful!

Confirm the app works on Cloud Run identically to that on App Engine. Your code should now match what's in the Module 5 repo folder. Congrats for completing this Module 5 codelab.

Optional: Clean up

What about cleaning up to avoid being billed until you're ready to move onto the next migration codelab? Since you're now using a different product, be sure to review the Cloud Run pricing guide.

Optional: Disable service

If you're not ready to go to the next tutorial yet, disable your service to avoid incurring additional charges. When you're ready to move onto the next codelab, you can re-enable it. While your app is disabled, it won't get any traffic to incur charges, however another thing you can get billed for is your Datastore usage if it exceeds the free quota, so delete enough to fall under that limit.

On the other hand, if you're not going to continue with migrations and want to delete everything completely, you can either delete your service or shutdown your project entirely.

Next steps

Congratulations... your app is now containrized, concluding this tutorial. From here, the next step is to learn about how to do the same thing with Docker in Module 4 codelab (link below) if you haven't already done so. There are other App Engine migrations to consider:

  • Migrate to Python 3 if you haven't already. The sample app is already 2.x & 3.x compatible, so the only change is for Docker users: update Dockerfile to use a Python 3 image; Buildpacks auto-detects language and selects appropriate base iamge.
  • Module 4: Migrate to Cloud Run with Docker
    • Containerize your app to run on Cloud Run with Docker
    • Allows you to stay on Python 2
  • Module 7: App Engine Push Task Queues (required if you use [push] Task Queues)
    • Adds App Engine taskqueue push tasks to Module 1 app
    • Prepares users for migrating to Cloud Tasks in Module 8
  • Module 3:
    • Modernize Datastore access from Cloud NDB to Cloud Datastore
    • This is the library used for Python 3 App Engine apps and non-App Engine apps
  • Module 6: Migrate to Cloud Firestore
    • Migrate to Cloud Firestore to access Firebase features
    • While Cloud Firestore supports Python 2, this codelab is available only in Python 3.

App Engine migration module codelabs issues/feedback

If you find any issues with this codelab, please search for your issue first before filing. Links to search and create new issues:

Migration resources

Links to the repo folders for Module 2 & 3 (START) and Module 5 (FINISH) can be found in the table below. They can also be accessed from the repo for all App Engine codelab migrations.

Codelab

Python 2

Python 3

Module 2

repo

(repo)

Module 3

(repo)

repo

Module 5

(n/a)

repo

App Engine & Cloud Run resources

Below are additional resources regarding this specific migration: