A local setup of a testing infrastructure vs a setup on Firebase Test Lab
Firebase Test Lab provides cloud-based infrastructure for testing Android apps. With one operation, you can initiate testing of your app across a wide variety of devices and device configurations. Test results — including logs, videos, and screenshots — are made available in your project in the Firebase Console. Even if you haven't written any test code for your app, Firebase Test Lab can exercise your app automatically, looking for crashes.
In this codelab, you'll learn how to get started with using Firebase Test Lab to test your mobile apps. You'll run your tests from the web, Android studio, or using the "gcloud" command line interface. You can even integrate it to your Continuous Integration environment!
The first step to use Firebase Test Lab is to navigate to your project page in the Firebase console, and upgrade your Firebase account to the Blaze tier. This allows you to access various Firebase capabilities including Firebase Test Lab. To upgrade your account to the Blaze tier, click on the upgrade button on the lower left of the navigation panel and select the Blaze plan.
You can either download the sample code into your computer...
...or clone the GitHub repository from the command line.
$ git clone https://github.com/googlesamples/android-testing.git
$ ./gradlew :app:assembleDebug $ ./gradlew :app:assembleDebugAndroidTest
$ ls app/build/outputs/apk/
The easiest way to get started with Firebase Test Lab is by running your tests on the web. All you need to start your first test is the application's APK. To run your first test, you can do the following:
For this type of test, all you need is the app's APK. Nothing else is needed to use Robo test. Robo test crawls your app and attempts to find crashes and helps you to identify UI discrepancies between multiple device models.
Note: Not all devices support all Android API levels. When you select a combination of devices and API levels that includes unsupported combinations, Test Lab will notify you that these combinations are unsupported, and those combinations are skipped when you start your tests.
In just a few clicks, with nothing but the app's APK, we were able to start our tests on a wide variety of devices and configurations, using infrastructure that would've otherwise taken a very long time to build and maintain, while providing test coverage on devices you might not have access to.
Now let's integrate Test Lab into our development environment by using it during Development from Android Studio or during code check-in from our Continuous Integration environments (e.g. Jenkins).
Android Studio should be installed on your codelab machine. If you're following this codelab on your personal machine, make sure you download and install Android Studio 2.0+ from here.
With the code downloaded, the following instructions describe how to open the completed sample app in Android Studio.
BasicSampledirectory from the sample code folder (Quickstart > Import Project... >
Now that we've used Firebase Test Lab from Android Studio and ensured that our code works during development, let's take it one step further and use it from the command-line, which is the first step towards a more formal Continuous integration with our deployment workflow.
$ gcloud components install beta
$ gcloud config set project <your firebase project name>
$ gcloud auth login
$ gcloud beta test android devices list
$ gcloud beta test android run \ --type instrumentation \ --app app/build/outputs/apk/app-debug-unaligned.apk \ --test app/build/outputs/apk/app-debug-androidTest-unaligned.apk \ --device-ids flounder,hammerhead \ --os-version-ids 19,21 \ --locales en,fr \ --orientations portrait,landscape
$ gcloud beta test android devices list
$ gcloud beta test android run \ --type instrumentation \ --app app/build/outputs/apk/app-debug-unaligned.apk \ --test app/build/outputs/apk/app-debug-androidTest-unaligned.apk \ --device-ids k3g \ --os-version-ids 19 \ --locales en,fr \ --orientations portrait,landscape
Using a service account tells gcloud that the user is a robot. This avoids checks for spam and prevents having the account mistakenly blocked or prompted for captchas.
You have now tried different uses for Firebase Test Lab and passed a major milestone towards achieving better quality for your app!