In this codelab, we will walk you through how to test your Unity project using the Firebase Test Lab for Android's Game Loop into your own project. We will use Unity's Survival Shooter example project as a starting point, but feel free to follow the steps with your own project.

What you'll learn

Tell us a little about yourself

How will you use this codelab?

Only read through it Read it and complete the exercises

How would you rate your experience with building Unity applications?

Never tried Novice Intermediate Proficient

How would you rate your experience with Firebase Test Lab?

Never tried Novice Intermediate Proficient

How would you rate your experience writing code?

Never tried Novice Intermediate Proficient

Hardware you'll need

Software you'll need

Enable Android USB Debugging on your device

In order to run samples, you'll need to enable debugging over USB on your device. To do so:

  1. Go to Settings
  2. Go to About phone (usually at the bottom)
  3. Tap Build number (usually at the bottom) seven times. You should see a toast saying you are now a developer.
  4. Return to the previous screen to find Developer options at the bottom.
  5. From the Developer options menu, find USB debugging and turn it on.
  6. When you connect your phone to your development platform, you will need to authenticate debugging from the phone. Allowing this will enable your machine to control your device.

Support Code

We've provided a .unitypackage with support code. You can download it from GitHub.

Get the project

You can either download the starter project to your computer...

Download Project .unitypackage

...or clone the GitHub repository from the command line.

git clone https://github.com/googlecodelabs/unity-firebase-test-lab-game-loop.git

To get started, we'll need to create a new project from scratch. Go ahead and launch Unity.

  1. After unity boots up, you will be greeted with a welcome dialog. Click the New button at the top to setup a new project.
  2. Set the Project name to "unity-firebase-test-lab-game-loop"
  3. You can set the Location to whatever you prefer.

If this is your first time using Unity, or you are still a beginner, let's walk through some important parts of Unity.

Using the Default Layout

First off let's use the default layout for the editor. At the top right of the editor, you will see several drop downs. Click the Layout dropdown and select Default. This will arrange the editor windows for you.

Unity's Windows

In Unity, there's several windows you should be aware of. Feel free to explore the provided links, but don't worry if it feels like a lot, we'll walk you through some of the first time steps.

Building for Android

First thing we need to do is configure Unity to build the project for Android

  1. Click on File -> Build Settings
  2. Change the target platform by selecting Android under the Platform menu.
  3. Click on Switch Platform.
  4. If everything was setup correctly, it will look like this (note the Unity symbol to the right of Android).

First thing we need to do is download the assets for the Survival Shooter sample project that Unity provides through the AssetStore.

  1. From the Menu, Choose Window -> Asset Store. You will now see the Asset Store window focused in the editor.
  2. In the search bar, type in "Survival Shooter" and hit enter.

  1. Click on the search result for "Survival Shooter tutorial"

  1. You will be taken to the page for the Survival Shooter asset. Click the Download button.

  1. Accept the License Agreement and click "Accept" to proceed.

  1. Since this is a complete project, it will overwrite your project settings. To proceed, click "Import" to begin the import process.

  1. Next you will see the import dialog. click "Import" to start importing assets.

  1. Unity will begin importing assets. Wait for the status window to complete and go away.

As a quick test, we'll make sure the game will run in the editor.

  1. In the Project Window, click the _Complete-Game folder.
  2. Double-click the _Complete-Game scene to open the scene in the editor.

  1. Ensure the game is running correctly by clicking the Play Button on the Toolbar.

  1. In the Game view, you should be able walk around with the WASD keys and shoot with the mouse.

Next we will build and deploy the game to the device.

Setting up the Project

  1. From the menu, select File > Build Settings. (In the Build Settings window, you may see a warning about color space.)
  2. Click Player Settings.

  1. In the Inspector window, ensure the Other Settings section is expanded.
  2. Ensure that Color Space is set to Gamma.
  3. Set the Package Name attribute to "com.example.firebase.testlab"

Building the Game

  1. Connect your phone to your computer via USB.
  2. From the menu, select File > Build Settings. In the Build Settings window, click On the Build and Run button at the bottom right.

  1. In the file selection dialog, specify your desired location for the resulting APK file. Unity will build, install, and run the apk on your device.
  1. You should now see the game running on your device

Before we move on, we'll want to import the support code for this project. If you haven't done so already, grab the latest version of the support code. The latest .unitypackage is available to download from GitHub here.

  1. Open the .unitypackage from Windows Explorer or macOS Finder.
  2. Unity will extract the package and prepare it's files for import. You'll get a confirmation dialog first that list all the files. Click Import to proceed.

The first thing we want to do is create some simple tests! We are going to make two types: one that reports boot time and another that reports the number of frames after a certain amount of time. Let's start by setting up the script and adding a component to the scene.

Creating the Script

  1. In the Project window, right click on the Scripts folder -> Create -> C# Script
  2. Name the script "TestLoops"

  1. In the Inspector window, verify your script was created

Adding the Testing Object

  1. Next we will add a testing object to the scene and attach the TestLoops script to it. From the Menu, Choose GameObject -> Create Empty. You will see a new GameObject in the Hierarchy window.

  1. In the Inspector window, rename the object "TestLoops"

  1. Click the Add Component button.
  2. Type in "Test Loops" and hit enter or click it to add it

  1. In the Test Loops component, double click the Script field to open up the scripting editor.

Now that you are in the coding editor, you will see an initial class template like so. Next let's flesh out the code!

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class TestLoops : MonoBehaviour {

// Use this for initialization
void Start () {

}

// Update is called once per frame
void Update () {

  }
}
  1. First thing we will need to do is import the provided helper library. Within the library is a class called TestLabManager, which will handle extracting the test parameter and outputting to the results file. At the top of TestLoops.cs, import the namespace:
using Firebase.TestLab;
  1. Next add a reference to the TestLabManager and some constants for each test we will handle. In the TestLoops class, add the following code
private TestLabManager _testLabManager;

private const int BootTimeTest = 1;

private const int FrameRateAfterOneSecondTest = 2;
private const int FrameRateAfterThreeSecondsTest = 3;
  1. We'll want to instantiate the TestLabManager in the start function. Note we will want to do in the Start function(as opposed to the constructor) because the JNI layer is not ready during construction, so it's best to do so in Start.
// Use this for initialization
void Start() {
  _testLabManager = TestLabManager.Instantiate();
}
  1. Next we'll get to the meat of our tests, within update. We will first check to see if there is a test scenario present. If so, we will switch on that value and change the behavior of the app. If it is our boot time test, we will simply report the time since startup and notify the testing harness the test is complete. Otherwise we'll call a helper function, which we will define next.
// Update is called once per frame
void Update() {
 // only change the behavior of the app if there is a scenario being tested
 if (!_testLabManager.IsTestingScenario) return;

 switch (_testLabManager.ScenarioNumber) {
   case BootTimeTest:
     // print out the result and complete
     _testLabManager.LogToResults("Time to boot: " + 
                                  Time.realtimeSinceStartup + 
                                  "\n");
     _testLabManager.NotifyHarnessTestIsComplete();
     break;
   case FrameRateAfterOneSecondTest:
     CheckFramerate(1.0f);
     break;
   case FrameRateAfterThreeSecondsTest:
     CheckFramerate(3.0f);
     break;
   default:
     throw new ArgumentOutOfRangeException();
 }
}
  1. Finally we will add the CheckFramerate(float) function, which simply prints the frame count after a given amount of time.
 private void CheckFramerate(float afterTime) {
   if (!(Time.time > afterTime)) return;

   const string s = "Number frames after {0} seconds: {1}\n";
   string output = string.Format(s, afterTime, Time.frameCount);

   _testLabManager.LogToResults(output);
   _testLabManager.NotifyHarnessTestIsComplete();
 }
}
  1. In the end, your class should look something like this:
using System;
using Firebase.TestLab;
using UnityEngine;

public class TestLoops : MonoBehaviour {
 private TestLabManager _testLabManager;

 private const int BootTimeTest = 1;

 private const int FrameRateAfterOneSecondTest = 2;
 private const int FrameRateAfterThreeSecondsTest = 3;

 void Start() {
   _testLabManager = TestLabManager.Instantiate();
 }

 // Update is called once per frame
 void Update() {
   // only change the behavior of the app if there is a scenario being tested
   if (!_testLabManager.IsTestingScenario) return;

   switch (_testLabManager.ScenarioNumber) {
     case BootTimeTest:
       // print out the result and complete
       _testLabManager.LogToResults("Time to boot: " +                        
                                    Time.realtimeSinceStartup + 
                                    "\n");
       _testLabManager.NotifyHarnessTestIsComplete();
       break;
     case FrameRateAfterOneSecondTest:
       CheckFramerate(1.0f);
       break;
     case FrameRateAfterThreeSecondsTest:
       CheckFramerate(3.0f);
       break;
     default:
       throw new ArgumentOutOfRangeException();
   }
 }

 private void CheckFramerate(float afterTime) {
   if (!(Time.time > afterTime)) return;

   const string s = "Number frames after {0} seconds: {1}\n";
   string output = string.Format(s, afterTime, Time.frameCount);

   _testLabManager.LogToResults(output);
   _testLabManager.NotifyHarnessTestIsComplete();
 }
}

Now that the code is fleshed out and the component is added to the scene, we'll need to expose the tests to the testing framework. To do so, we'll need to create a custom Android manifest file.

Creating the default AndroidManifest.xml file

In general, Unity handles the AndroidManifest.xml generation on it's own. However, for this project we will need to add our own. We can provide one by placing it in the directory Assets/Plugins/Android. More details are available here.

  1. In the Project window, right click the Assets folder -> Create -> Folder.
  2. Name the folder "Plugins"

  1. Right click again on the Plugins folder -> Create -> Folder
  2. Name the folder "Android"

  1. Right click on the Android folder and choose "Show In Explorer/Reveal in Finder"
  2. Using your preferred text editor create the file AndroidManifest.xml and fill it in like so. This is similar to the manifest file Unity generates for you.
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android" package="com.example.firebase.testlab" xmlns:tools="http://schemas.android.com/tools" android:versionName="1.0" android:versionCode="1" android:installLocation="preferExternal">
 <supports-screens android:smallScreens="true" android:normalScreens="true" android:largeScreens="true" android:xlargeScreens="true" android:anyDensity="true" />
 <application android:theme="@style/UnityThemeSelector" android:icon="@drawable/app_icon" android:label="@string/app_name" android:debuggable="false" android:isGame="true" android:banner="@drawable/app_banner">
   <activity android:name="com.unity3d.player.UnityPlayerActivity" android:label="@string/app_name" android:screenOrientation="landscape" android:launchMode="singleTask" android:configChanges="mcc|mnc|locale|touchscreen|keyboard|keyboardHidden|navigation|orientation|screenLayout|uiMode|screenSize|smallestScreenSize|fontScale|layoutDirection">
     <intent-filter>
       <action android:name="android.intent.action.MAIN" />
       <category android:name="android.intent.category.LAUNCHER" />
       <category android:name="android.intent.category.LEANBACK_LAUNCHER" />
     </intent-filter>
     <meta-data android:name="unityplayer.UnityActivity" android:value="true" />
   </activity>
 </application>
 <uses-sdk android:minSdkVersion="16" android:targetSdkVersion="25" />
 <uses-feature android:glEsVersion="0x00020000" />
 <uses-feature android:name="android.hardware.touchscreen" android:required="false" />
 <uses-feature android:name="android.hardware.touchscreen.multitouch" android:required="false" />
 <uses-feature android:name="android.hardware.touchscreen.multitouch.distinct" android:required="false" />
</manifest>

Adding the Firebase Test Lab Game Loop meta-data

Now that we have our default AndroidManifest.xml, we can build and run the game again. But nothing will be different! This is because we haven't added any hooks to the manifest yet. Next we will add the intent-filter needed to expose the test loop.

  1. When launching the game loop test, your game is triggered with a specific intent, so you need to modify your manifest and add a new intent filter to your activity. In AndroidManifest.xml, add the following intent-filter to the primary activity element.
<intent-filter>
  <action android:name="com.google.intent.action.TEST_LOOP"/>
  <category android:name="android.intent.category.DEFAULT"/>
  <data android:mimeType="application/javascript"/>
</intent-filter>
        
  1. Next we will want to indicate how many tests are exposed. In this case we will make 3. Add the following meta-data to the manifest in the application element.
<meta-data android:name="com.google.test.loops" android:value="3"/>
  1. Now that we have three tests, let us add some labels for them. Our first test will evaluate the boot time. The other two will evaluate framerate. Add the following meta-data to the manifest in the application element. (For more information about labels, check here). Note that labels test match up to the constants we added to the TestLoops.cs file earlier.
<meta-data android:name="com.google.test.loops.boot.time" android:value="1" />
<meta-data android:name="com.google.test.loops.fps" android:value="2-3" />
  1. In the end, your final manifest file should look like this:
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android" package="com.example.firebase.testlab" xmlns:tools="http://schemas.android.com/tools" android:versionName="1.0" android:versionCode="1" android:installLocation="preferExternal">
  <supports-screens android:smallScreens="true" android:normalScreens="true" android:largeScreens="true" android:xlargeScreens="true" android:anyDensity="true" />
  <application android:theme="@style/UnityThemeSelector" android:icon="@drawable/app_icon" android:label="@string/app_name" android:debuggable="false" android:isGame="true" android:banner="@drawable/app_banner">
    <activity android:name="com.unity3d.player.UnityPlayerActivity" android:label="@string/app_name" android:screenOrientation="landscape" android:launchMode="singleTask" android:configChanges="mcc|mnc|locale|touchscreen|keyboard|keyboardHidden|navigation|orientation|screenLayout|uiMode|screenSize|smallestScreenSize|fontScale|layoutDirection">
      <intent-filter>
        <action android:name="android.intent.action.MAIN" />
        <category android:name="android.intent.category.LAUNCHER" />
        <category android:name="android.intent.category.LEANBACK_LAUNCHER" />
      </intent-filter>
       
           <intent-filter>
        <action android:name="com.google.intent.action.TEST_LOOP"/>
        <category android:name="android.intent.category.DEFAULT"/>
        <data android:mimeType="application/javascript"/>
      </intent-filter>
          
      <meta-data android:name="unityplayer.UnityActivity" android:value="true" />
    </activity>   
    
        <meta-data android:name="com.google.test.loops" android:value="3"/>

    <meta-data android:name="com.google.test.loops.boot.time" android:value="1" />
    <meta-data android:name="com.google.test.loops.fps" android:value="2-3" />
   </application>
   
   <uses-sdk android:minSdkVersion="16" android:targetSdkVersion="25" />
   <uses-feature android:glEsVersion="0x00020000" />
   <uses-feature android:name="android.hardware.touchscreen" android:required="false" />
   <uses-feature android:name="android.hardware.touchscreen.multitouch" android:required="false" />
   <uses-feature android:name="android.hardware.touchscreen.multitouch.distinct" android:required="false" />
</manifest>
  1. Finally build and run your application. You shouldn't notice a difference, but that's because the game isn't launching with the testing intent. Next up we'll start using the latest Test Loop Manager to launch the game with our new testing intent.

Now that we have our intents exposed via the manifest file, and our tests are written, we can start running the tests!. To do so, we will use the provided Test Loop Manager application.

  1. If you haven't done so already, download the latest The latest Test Loop Manager apk.
  2. Use adb to install the apk to your device.
adb install testloopmanager.apk
  1. Run the Test Loop Manager on your device.

  1. You will see the Nightmares listed as an application with Test Loops. Tap on it.

  1. You will see the lists of test available. You can pick and choose which ones to run from this window. Ensure all tests are selected.
  2. Tap "Run 3 Test Loops"

Verifying the results

Once all the tests are done running, we can check the results of them by using adb to print the results.

  1. Use adb to establish a shell with the device.
adb shell
  1. Change the directory to the sdcard, where Test Loop Manager stores the output of the tests.
cd /sdcard
  1. Look at the results file in the directory by using the following command:
tail *.json
  1. You should see results like so:
==> results1.json <==
Time to boot: 2.801764

==> results2.json <==
Number frames after 1 seconds: 32

==> results3.json <==
Number frames after 3 seconds: 91

As you can see, you can get some pretty useful information here, and make as many tests as you like. Finally, we'll push this build to the cloud and get the tests running on multiple devices!

  1. First we'll need to create a Firebase project start by going to Firebase console here. If you have not already, log in with a valid Google account.
  2. Click Add project

  1. In the project name field, type "Nightmares"
  2. Click Create Project

  1. In the left hand menu, click Test Lab
  2. Click Run your first test
  3. Choose Game Loop as your test type.
  4. Click Continue

  1. Click Browse and upload the apk you built via Unity.
  2. In the Scenarios field, type in "1-3"
  3. Click Continue

  1. Next you will be presented with a list of devices. Choose Pixel(API Level 25 & 26) for a newer device.
  2. Also choose Nexus 5(API Level 23) for an older device.

  1. Scroll to the bottom and set Orientation to just Landscape.
  2. Click "Start 3 Tests" to start the tests.

  1. Now all we have to do is wait for the tests to complete. This can take a while, so hang in there!

Once the tests are completed running, you should see some green check marks. Let's dive into some results!

  1. Click on the Nexus 5, API Level 23.

  1. You will now see the results files from the tests. Expand the results to see the details. It should resemble the results you saw above when run locally.

You did it! You've added some tests to a simple app and seen the various ways to see the results of those tests!

Beyond

If you are looking for some more interesting things to do try:

Additionally, if you want to explore the Firebase Test Lab platform further, check out the documentation here.

Tell us how it went!

Were you able to complete the codelab?

Yes! Yes, with some issues. Nope!

What did you think about the amount of code?

Just Right Too Little Too Much

What did you think about the amount of Unity setup?

Just Right Too Little Too Much

Overall, are you glad you did this codelab?

Yes! No...