The Google App Engine (GAE) migration modules show App Engine (Standard) developers how to modernize their apps running on the original legacy runtimes to the next generation platform. Once that's accomplished, they can make their apps more portable by explicitly containerizing them for Cloud Run, Google Cloud's container-hosting sister service to App Engine, and other container-hosting services.
This tutorial teaches you how to containerize App Engine apps for deploying to the Cloud Run fully-managed service using Docker, a well-known platform in industry for developing, shipping, and running applications in containers. For Python 2 developers, this tutorial STARTs with the Module 2 Cloud NDB App Engine sample app while Python 3 developers START with the Module 3 Cloud Datastore sample.
You'll learn how to
- Containerize your app using Docker
- Deploy container images to Cloud Run
What you'll need
- A Google Cloud Platform project with:
- Basic Python skills
- Working knowledge of common Linux commands
- Basic knowledge of developing & deploying App Engine apps
- A working App Engine app ready to be containerized
How will you use this codelab?
PaaS systems like App Engine and Cloud Functions provide conveniences of serverless platforms like focus away from SysAdmin/DevOps towards building solutions, autoscaling up as needed, scaling down to 0 and pay-per-use billing to control costs, and variety of common development languages. However, the flexibility of containers is compelling as well, the ability to choose any language, any library, any binary. Giving users the best of both worlds, the convenience of serverless along with the flexibility of containers, is what Google Cloud Run is all about.
Learning how to use Cloud Run is not within the scope of this tutorial; that's covered by the Cloud Run documentation. By the end of this tutorial, you'll know how to containerize your App Engine app for Cloud Run (or other services). There are a few things you should know before moving forward, primarily that your user experience will be slightly different, a bit lower-level as you will no longer be taking application code and deploying it.
Instead, you'll need to learn something about containers like how to build and deploy them. You also get to decide what you want to put in the container image, including a web server as you won't be using App Engine's web server any more. If your prefer not to follow this path, keeping your apps on App Engine is not a bad option.
In this tutorial, you'll learn how to containerize your app, replacing App Engine config files with container configuration, determine what goes into the container, then specify how to start your app — many of these things are automatically handled by App Engine.
This migration features these steps:
- Containerize application
- Replace configuration files
- Modify application files
Before we get going with the main part of the tutorial, let's setup our project, get the code, then deploy the baseline app so we know we started with working code.
1. Setup project
If you completed either the Module 2 or Module 3 codelabs, we recommend reusing that same project (and code). Alternatively, you can create a brand new project or reuse another existing project. Ensure the project has an active billing account and App Engine (app) is enabled.
2. "Get" baseline sample app
One of the prerequisites to this codelab is to have a working Module 2 or Module 3 sample app. If you don't have one, go complete either tutorial (links above) before moving ahead here. Otherwise if you're already familiar with its contents, you can just start by grabbing the Module 2 or 3 code below.
Whether you use yours or ours, the Module 2 code is where we'll START for the Python 2 version of this tutorial, and similarly, the Module 3 code for Python 3. This Module 4 codelab walks you through each step, and depending on your options, you should end up with code that resembles one of the Module 4 repo folders (FINISH) when complete.
- Python 2 (Cloud NDB app)
- Python 3 (Cloud Datastore app)
- Entire repo (to clone or download ZIP)
The directory of Python 2 Module 2 STARTing files (yours or ours) should look like this:
$ ls README.md appengine_config.py requirements.txt app.yaml main.py templates
If you're using your own Module 2 (2.x) code, you'll also have a
lib folder. Neither
appengine_config.py are used for Python 3, where the Module 3 (3.x) STARTing code should look like this:
$ ls README.md main.py templates app.yaml requirements.txt
3. (Re)Deploy baseline app
Your remaining prework steps to execute now:
- Re-familiarize yourself with the
- Re-deploy sample app with
gcloud app deploy
- Confirm app runs on App Engine without issue
Once you've successfully executed those steps, you're ready to containerize it.
Docker is the standard containerization platform in industry today. One challenge in using it, as mentioned earlier, is that it requires effort to curating an efficient
Dockerfile, the configuration file that determines how your container images are built. The Buildpacks open specification lift this burden off the developer, using introspection to determine your app's dependencies, and builds the most efficient container for your app as possible.
You're in the right place if you already know about containers, Docker, and want to learn more about containerizing your App Engine app for Cloud Run. Feel free to also do the Module 5 codelab (identical to this one but with Cloud Buildpacks) afterwards. Our barebones sample app is lightweight enough to avoid some of those aforementioned
The migration steps include replacing the App Engine configuration files and specifying how your app should start. Below is a table summarizing the configuration files to expect for each platform type:
Migrating from App Engine means replacing
app.yaml with a
Dockerfile which outlines how to build & run the container. App Engine automatically starts your application, but Cloud Run doesn't. This is what the
CMD commands are for. Learn more about
Dockerfile from this Cloud Run docs page as well as see an example
Dockerfile that spawns
An alternative to using
CMD in a
Dockerfile is to use a
Procfile. Finally, a
.dockerignore helps filter out non-app files to keep your container size down. More on these coming up!
app.yaml and create
file is not used in containers so delete it now.
The container configuration file is
Dockerfile, and our sample app only requires a minimal one. Create your
Dockerfile by copying the following into yours, replacing
NNN with "
2" or "
3", depending on which Python version you're using.
FROM python:NNN-slim WORKDIR /app COPY . . RUN pip install -r requirements.txt ENTRYPOINT ["python", "main.py"]
Most of the
Dockerfile specifies how to create the container while the
ENTRYPOINT command starts the container; in this case calling
python main.py to start the Flask development server. If you're new to Docker, the
FROM directive indicates the base image to start from, and "slim" refers to a minimal Python distribution. Learn more from the Docker Python images page.
The middle set of commands creates the working directory (
/app), copies in the application files, then runs
pip install to bring third-party libraries into the container.
WORKDIR combines the Linux
cd commands together; read more about it in the
WORKDIR documentation . The
RUN directives are self-explanatory.
requirements.txt file can stay the same; Flask should be there along with your Datastore client library (Cloud Datastore or Cloud NDB). If you wish to use another WSGI-compliant HTTP server like Gunicorn — current version at the time of this writing is 20.0.4 — then add
Python 2 App Engine developers know that third-party libraries are either copied into the
lib folder, referenced in
requirements.txt, itemized in
app.yaml, and supported by
appengine_config.py. Containers, like Python 3 App Engine apps, only use
requirements.txt, so all the other stuff can be dropped, meaning if you have a 2.x App Engine app, delete
Python 2 users do not startup App Engine's web server, but Python 3 users have the option of converting their
app.yaml files to have an
entrypoint instead of
script: auto directives in their
handlers section. If you use
entrypoint in your Python 3
app.yaml, it would look something like this:
runtime: python38 entrypoint: python main.py
entrypoint directive tells App Engine how to start your server. You can move it almost directly into your
Dockerfile; and similarly, such a line would go into
Procfile if you used Buildpacks:
- Docker: This line in
ENTRYPOINT ["python", "main.py"]
- Buildpacks: A
Procfileconsisting of just this line:
web: python main.py
Using the Flask development server is fine for testing, but if using a production server like
gunicorn for your application, be sure to point your
CMD directive at it like in the Cloud Run Quickstart sample.
We recommend creating a
.dockerignore file to trim the size of your container and not clutter your container image with superfluous files like these:
*.md *.pyc *.pyo .git/ .gitignore __pycache__
Whether you have a Module 2 or Module 3 app, all are fully 2.x & 3.x compatible, meaning there are no changes to the core components of
main.py; we will only be adding a few lines of startup code. Most required changes are in configuration.
Add a pair of lines at the bottom of
main.py to start the application. Cloud Run automatically "injects" 8080 as the
PORT environment variable, so you don't need to set it in
Cloud Run starts its web server on port 8080, automatically injected into the
PORT environment variable. Add an
import os at the top of
main.py as well as a "main" at the bottom to start the server (if executed directly which our
if __name__ == '__main__': import os app.run(debug=True, threaded=True, host='0.0.0.0', port=int(os.environ.get('PORT', 8080)))
With your App Engine configuration replaced by either Docker's, you're ready to deploy.
1. Build Docker image & send to Cloud Container Registry
Execute this command to build your container image and archive to the Cloud Registry:
$ gcloud builds submit --tag gcr.io/PROJECT_ID/IMG_NAME Creating temporary tarball archive of 5 file(s) totalling 1.9 KiB before compression. Uploading tarball of [.] to [gs://PROJECT_ID_cloudbuild/source/1605604100.565976-8b4e2297826343d3bc21c36fbd3736b8.tgz] Created [https://cloudbuild.googleapis.com/v1/projects/PROJECT_ID/builds/8c3900e2-ca90-441f-bf29-b1cfd5cab309]. Logs are available at [https://console.cloud.google.com/cloud-build/builds/8c3900e2-ca90-441f-bf29-b1cfd5cab309?project=1046901301037]. ------------------------------------ REMOTE BUILD OUTPUT ------------------------------------ starting build "8c3900e2-ca90-441f-bf29-b1cfd5cab309" FETCHSOURCE Fetching storage object: gs://PROJECT_ID_cloudbuild/source/1605604100.565976-8b4e2297826343d3bc21c36fbd3736b8.tgz#1605604101281977 Copying gs://PROJECT_ID_cloudbuild/source/1605604100.565976-8b4e2297826343d3bc21c36fbd3736b8.tgz#1605604101281977... / [1 files][ 1.3 KiB/ 1.3 KiB] Operation completed over 1 objects/1.3 KiB. BUILD Already have image (with digest): gcr.io/cloud-builders/docker Sending build context to Docker daemon 7.68kB Module 1/5 : FROM python:3-slim 3-slim: Pulling from library/python bb79b6b2107f: Already exists 35e30c3f3e2b: Pulling fs layer b13c2c0e2577: Pulling fs layer 263be93302fa: Pulling fs layer 30e7021a7001: Pulling fs layer 30e7021a7001: Waiting 263be93302fa: Verifying Checksum 263be93302fa: Download complete 35e30c3f3e2b: Verifying Checksum 35e30c3f3e2b: Download complete b13c2c0e2577: Verifying Checksum b13c2c0e2577: Download complete 35e30c3f3e2b: Pull complete 30e7021a7001: Verifying Checksum 30e7021a7001: Download complete b13c2c0e2577: Pull complete 263be93302fa: Pull complete 30e7021a7001: Pull complete Digest: sha256:c13fda093489a1b699ee84240df4f5d0880112b9e09ac21c5d6875003d1aa927 Status: Downloaded newer image for python:3-slim ---> a90139e6bc2f Module 2/5 : WORKDIR /app ---> Running in 17fa98a79d4a Removing intermediate container 17fa98a79d4a ---> cf1f7de92bb0 Module 3/5 : COPY . . ---> a4703449acd6 Module 4/5 : RUN pip install -r requirements.txt ---> Running in e23df87afa98 Collecting Flask==1.1.2 Downloading Flask-1.1.2-py2.py3-none-any.whl (94 kB) Collecting google-cloud-datastore==2.0.0 Downloading google_cloud_datastore-2.0.0-py2.py3-none-any.whl (144 kB) Collecting itsdangerous>=0.24 Downloading itsdangerous-1.1.0-py2.py3-none-any.whl (16 kB) Collecting Jinja2>=2.10.1 Downloading Jinja2-2.11.2-py2.py3-none-any.whl (125 kB) Collecting Werkzeug>=0.15 Downloading Werkzeug-1.0.1-py2.py3-none-any.whl (298 kB) Collecting click>=5.1 Downloading click-7.1.2-py2.py3-none-any.whl (82 kB) Collecting libcst>=0.2.5 Downloading libcst-0.3.13-py3-none-any.whl (502 kB) Collecting google-api-core[grpc]<2.0.0dev,>=1.22.2 Downloading google_api_core-1.23.0-py2.py3-none-any.whl (91 kB) Collecting google-cloud-core<2.0dev,>=1.4.0 Downloading google_cloud_core-1.4.3-py2.py3-none-any.whl (27 kB) Collecting proto-plus>=1.4.0 Downloading proto-plus-1.11.0.tar.gz (44 kB) Collecting MarkupSafe>=0.23 Downloading MarkupSafe-1.1.1.tar.gz (19 kB) Collecting pyyaml>=5.2 Downloading PyYAML-5.3.1.tar.gz (269 kB) Collecting typing-inspect>=0.4.0 Downloading typing_inspect-0.6.0-py3-none-any.whl (8.1 kB) Collecting typing-extensions>=184.108.40.206 Downloading typing_extensions-220.127.116.11-py3-none-any.whl (22 kB) Collecting requests<3.0.0dev,>=2.18.0 Downloading requests-2.25.0-py2.py3-none-any.whl (61 kB) Collecting googleapis-common-protos<2.0dev,>=1.6.0 Downloading googleapis_common_protos-1.52.0-py2.py3-none-any.whl (100 kB) Requirement already satisfied: setuptools>=34.0.0 in /usr/local/lib/python3.9/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.22.2->google-cloud-datastore==2.0.0->-r requirements.txt (line 2)) (50.3.2) Collecting six>=1.13.0 Downloading six-1.15.0-py2.py3-none-any.whl (10 kB) Collecting pytz Downloading pytz-2020.4-py2.py3-none-any.whl (509 kB) Collecting google-auth<2.0dev,>=1.21.1 Downloading google_auth-1.23.0-py2.py3-none-any.whl (114 kB) Collecting protobuf>=3.12.0 Downloading protobuf-3.14.0-py2.py3-none-any.whl (173 kB) Collecting grpcio<2.0dev,>=1.29.0; extra == "grpc" Downloading grpcio-1.33.2-cp39-cp39-manylinux2014_x86_64.whl (3.8 MB) Collecting mypy-extensions>=0.3.0 Downloading mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB) Collecting chardet<4,>=3.0.2 Downloading chardet-3.0.4-py2.py3-none-any.whl (133 kB) Collecting certifi>=2017.4.17 Downloading certifi-2020.11.8-py2.py3-none-any.whl (155 kB) Collecting idna<3,>=2.5 Downloading idna-2.10-py2.py3-none-any.whl (58 kB) Collecting urllib3<1.27,>=1.21.1 Downloading urllib3-1.26.2-py2.py3-none-any.whl (136 kB) Collecting pyasn1-modules>=0.2.1 Downloading pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB) Collecting cachetools<5.0,>=2.0.0 Downloading cachetools-4.1.1-py3-none-any.whl (10 kB) Collecting rsa<5,>=3.1.4; python_version >= "3.5" Downloading rsa-4.6-py3-none-any.whl (47 kB) Collecting pyasn1<0.5.0,>=0.4.6 Downloading pyasn1-0.4.8-py2.py3-none-any.whl (77 kB) Building wheels for collected packages: proto-plus, MarkupSafe, pyyaml Building wheel for proto-plus (setup.py): started Building wheel for proto-plus (setup.py): finished with status 'done' Created wheel for proto-plus: filename=proto_plus-1.11.0-py3-none-any.whl size=41569 sha256=54b12f4c68d426bdd21a73ec422e5df7351f09cd45ad673b93ec13b0a8b8a1ba Stored in directory: /root/.cache/pip/wheels/d8/b4/72/1293fbaaa4e6deb58b262d1d0b7aa9713b5a580784d373be0d Building wheel for MarkupSafe (setup.py): started Building wheel for MarkupSafe (setup.py): finished with status 'done' Created wheel for MarkupSafe: filename=MarkupSafe-1.1.1-py3-none-any.whl size=12627 sha256=ddc8a1e4a44df58258de3f31dc13ae3ad179afbf97eb264c8c190d734498766d Stored in directory: /root/.cache/pip/wheels/e0/19/6f/6ba857621f50dc08e084312746ed3ebc14211ba30037d5e44e Building wheel for pyyaml (setup.py): started Building wheel for pyyaml (setup.py): finished with status 'done' Created wheel for pyyaml: filename=PyYAML-5.3.1-cp39-cp39-linux_x86_64.whl size=44617 sha256=8e2901e9bb4fc3b9b505bf69cecf826401a3e7ecf7525bb57d63b30804721144 Stored in directory: /root/.cache/pip/wheels/69/60/81/5cd74b8ee068fbe9e04ca0d53148f28f5c6e2c5b177d5dd622 Successfully built proto-plus MarkupSafe pyyaml Installing collected packages: itsdangerous, MarkupSafe, Jinja2, Werkzeug, click, Flask, pyyaml, mypy-extensions, typing-extensions, typing-inspect, libcst, chardet, certifi, idna, urllib3, requests, six, protobuf, googleapis-common-protos, pytz, pyasn1, pyasn1-modules, cachetools, rsa, google-auth, grpcio, google-api-core, google-cloud-core, proto-plus, google-cloud-datastore Successfully installed Flask-1.1.2 Jinja2-2.11.2 MarkupSafe-1.1.1 Werkzeug-1.0.1 cachetools-4.1.1 certifi-2020.11.8 chardet-3.0.4 click-7.1.2 google-api-core-1.23.0 google-auth-1.23.0 google-cloud-core-1.4.3 google-cloud-datastore-2.0.0 googleapis-common-protos-1.52.0 grpcio-1.33.2 idna-2.10 itsdangerous-1.1.0 libcst-0.3.13 mypy-extensions-0.4.3 proto-plus-1.11.0 protobuf-3.14.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pytz-2020.4 pyyaml-5.3.1 requests-2.25.0 rsa-4.6 six-1.15.0 typing-extensions-18.104.22.168 typing-inspect-0.6.0 urllib3-1.26.2 Removing intermediate container e23df87afa98 ---> a94cb28e33e9 Module 5/5 : ENTRYPOINT ["python", "main.py"] ---> Running in fd5bd0bfa781 Removing intermediate container fd5bd0bfa781 ---> f8dc8ff328c1 Successfully built f8dc8ff328c1 Successfully tagged gcr.io/PROJECT_ID/IMG_NAME:latest PUSH Pushing gcr.io/PROJECT_ID/IMG_NAME The push refers to repository [gcr.io/PROJECT_ID/IMG_NAME] cd9b381d5582: Preparing . . . d0fe97fa8b8c: Preparing 83dcc420d3e6: Waiting 225ef82ca30a: Waiting d0fe97fa8b8c: Waiting 8414f9cd7a8a: Pushed c5deba9fb608: Pushed d93c07024f51: Pushed 225ef82ca30a: Layer already exists d0fe97fa8b8c: Layer already exists b362843246c4: Pushed cd9b381d5582: Pushed 83dcc420d3e6: Pushed latest: digest: sha256:532c3ba8f6f8747ff43437483a92aa46080a960627f0ac413e5cea2524797ce4 size: 1997 DONE --------------------------------------------------------------------------------------------- ID CREATE_TIME DURATION SOURCE IMAGES STATUS 8c3900e2-ca90-441f-bf29-b1cfd5cab309 2020-11-17T09:08:21+00:00 34S gs://PROJECT_ID_cloudbuild/source/1605604100.565976-8b4e2297826343d3bc21c36fbd3736b8.tgz gcr.io/PROJECT_ID/IMG_NAME (+1 more) SUCCESS
2. Deploy Docker image from Container Registry
Execute this command to deploy that image to run on Cloud Run:
$ gcloud run deploy SVC_NAME --platform managed --image gcr.io/PROJECT_ID/IMG_NAME Please specify a region:  asia-east1  asia-east2  asia-northeast1  asia-northeast2  asia-northeast3  asia-south1  asia-southeast1  asia-southeast2  australia-southeast1  europe-north1  europe-west1  europe-west2  europe-west3  europe-west4  europe-west6  northamerica-northeast1  southamerica-east1  us-central1  us-east1  us-east4  us-west1  cancel Please enter your numeric choice: 21 To make this the default region, run `gcloud config set run/region us-west1`. Deploying container to Cloud Run service [SVC_NAME] in project [PROJECT_ID] region [us-west1] ✓ Deploying... Done. ✓ Creating Revision... ✓ Routing traffic... Done. Service [SVC_NAME] revision [SVC_NAME-00012-waj] has been deployed and is serving 100 percent of traffic. Service URL: https://SVC_NAME-REGION_HASH.a.run.app
Choose your region, and allow unauthenticated connections for easier testing.
Confirm the app works on Cloud Run identically to that on App Engine. Your code should now match what's in the Module 4 repo folder, whether it's 2.x or 3.x. Congrats for completing this Module 4 codelab.
Optional: Clean up
What about cleaning up to avoid being billed until you're ready to move onto the next migration codelab? Since you're now using a different product, be sure to review the Cloud Run pricing guide.
Optional: Disable service
If you're not ready to go to the next tutorial yet, disable your service to avoid incurring additional charges. When you're ready to move onto the next codelab, you can re-enable it. While your app is disabled, it won't get any traffic to incur charges, however another thing you can get billed for is your Datastore usage if it exceeds the free quota, so delete enough to fall under that limit.
Congratulations... your app is now containrized, concluding this tutorial. From here, the next step is to learn about how to do the same thing with Cloud Buildpacks in Module 5 codelab (link below) or other App Engine migrations to consider:
- Migrate to Python 3 if you haven't already. The sample app is already 2.x & 3.x compatible, so the only change is for Docker users: update
Dockerfileto use a Python 3 image; Buildpacks auto-detects language and selects appropriate base iamge.
- Module 5: Migrate to Cloud Run with Cloud Buildpacks
- Containerize your app to run on Cloud Run with Cloud Buildpacks
- Do not need to know anything about Docker, containers, or
- Requires you to have already migrated your app to Python 3
- Module 7: App Engine Push Task Queues (required if you use [push] Task Queues)
- Adds App Engine
taskqueuepush tasks to Module 1 app
- Prepares users for migrating to Cloud Tasks in Module 8
- Adds App Engine
- Module 3:
- Modernize Datastore access from Cloud NDB to Cloud Datastore
- This is the library used for Python 3 App Engine apps and non-App Engine apps
- Module 6: Migrate to Cloud Firestore
- Migrate to Cloud Firestore to access Firebase features
- While Cloud Firestore supports Python 2, this codelab is available only in Python 3.
App Engine migration module codelabs issues/feedback
If you find any issues with this codelab, please search for your issue first before filing. Links to search and create new issues:
Links to the repo folders for Module 2 & 3 (START) and Module 4 (FINISH) can be found in the table below. They can also be accessed from the repo for all App Engine codelab migrations.
App Engine & Cloud Run resources
Below are additional resources regarding this specific migration: