A local setup of a testing infrastructure vs a setup on Firebase Test Lab

Firebase Test Lab provides cloud-based infrastructure for testing Android apps. With one operation, you can initiate testing of your app across a wide variety of devices and device configurations. Test results — including logs, videos, and screenshots — are made available in your project in the Firebase Console. Even if you haven't written any test code for your app, Firebase Test Lab can exercise your app automatically, looking for crashes.

In this codelab, you'll learn how to get started with using Firebase Test Lab to test your mobile apps. You'll run your tests from the web, Android studio, or using the "gcloud" command line interface. You can even integrate it to your Continuous Integration environment!

What you'll learn

What you'll need

Upgrade your Firebase Project to the Blaze tier

The first step to use Firebase Test Lab is to navigate to your project page in the Firebase console, and upgrade your Firebase account to the Blaze tier. This allows you to access various Firebase capabilities including Firebase Test Lab. To upgrade your account to the Blaze tier, click on the upgrade button on the lower left of the navigation panel and select the Blaze plan.

Get the sample code

You can either download the sample code into your computer...

Download Zip

...or clone the GitHub repository from the command line.

$ git clone https://github.com/googlesamples/android-testing.git

Get your app and test APKs

$ ./gradlew :app:assembleDebug
$ ./gradlew :app:assembleDebugAndroidTest
$ ls app/build/outputs/apk/

The easiest way to get started with Firebase Test Lab is by running your tests on the web. All you need to start your first test is the application's APK. To run your first test, you can do the following:

For this type of test, all you need is the app's APK. Nothing else is needed to use Robo test. Robo test crawls your app and attempts to find crashes and helps you to identify UI discrepancies between multiple device models.

Note: Not all devices support all Android API levels. When you select a combination of devices and API levels that includes unsupported combinations, Test Lab will notify you that these combinations are unsupported, and those combinations are skipped when you start your tests.

In just a few clicks, with nothing but the app's APK, we were able to start our tests on a wide variety of devices and configurations, using infrastructure that would've otherwise taken a very long time to build and maintain, while providing test coverage on devices you might not have access to.

Now let's integrate Test Lab into our development environment by using it during Development from Android Studio or during code check-in from our Continuous Integration environments (e.g. Jenkins).

Install Android Studio 2.0+

Android Studio should be installed on your codelab machine. If you're following this codelab on your personal machine, make sure you download and install Android Studio 2.0+ from here.

Import the sample project

With the code downloaded, the following instructions describe how to open the completed sample app in Android Studio.

  1. Select the BasicSample directory from the sample code folder (Quickstart > Import Project... > android-testing > ui > espresso > BasicSample).
  2. Install any missing packages and resolve suggested conflicts
  3. Click the Gradle sync button. Resolve all issues.
  4. If you encounter any issues, follow the suggestions from Android Studio to install missing packages until you completely resolve the issue.

Run a Firebase Test Lab test from Android Studio

Install gcloud command-line interface and enable Firebase Test Lab plugin

Now that we've used Firebase Test Lab from Android Studio and ensured that our code works during development, let's take it one step further and use it from the command-line, which is the first step towards a more formal Continuous integration with our deployment workflow.

$ gcloud components install beta
$ gcloud config set project <your firebase project name>
$ gcloud auth login
$ gcloud beta test android devices list

Run a Cloud Test from gcloud command-line interface

$ gcloud beta test android run \
  --type instrumentation \
  --app app/build/outputs/apk/app-debug-unaligned.apk \
  --test app/build/outputs/apk/app-debug-androidTest-unaligned.apk \
  --device-ids flounder,hammerhead \
  --os-version-ids 19,21 \
  --locales en,fr \
  --orientations portrait,landscape
$ gcloud beta test android devices list
$ gcloud beta test android run \
  --type instrumentation \
  --app app/build/outputs/apk/app-debug-unaligned.apk \
  --test app/build/outputs/apk/app-debug-androidTest-unaligned.apk \
  --device-ids k3g \
  --os-version-ids 19 \
  --locales en,fr \
  --orientations portrait,landscape

Prerequisites

Using a service account tells gcloud that the user is a robot. This avoids checks for spam and prevents having the account mistakenly blocked or prompted for captchas.

Add a build step to Jenkins

  1. Connect your Jenkins build to the github repository
  2. Add a trigger on new pull requests
  3. Add a build step to run the gradle build commands mentioned above
  4. Add a build step for the "gcloud test" commands mentioned above
  5. Start a new build and then watch to see if Jenkins marks the build as successfully tested, or if it finds a test failure.

You have now tried different uses for Firebase Test Lab and passed a major milestone towards achieving better quality for your app!

What we've covered

Next Steps

Tell us how we did