Performing Mass Instrumentation Tests with Firebase Test Lab for Android (gCloud Command Line)

Android

Firebase Test Lab’s most prominent feature is almost definitely the Robo Test, but is that the most powerful? I’d beg to differ.

While the robo test is handy for creating quick tests that simulate actual user interaction, just like a user’s navigation around the app, it could be pretty random. There could easily be sections of your app left untested.

In comes the Instrumentation Test. The defining quality of an instrumentation test is that you write the test yourself. If you know anything about Test Driven Development (TDD), you know this is the way to a nice stable app since you can define specific tests for each part of your app.

Let’s get some facts out of the way. Instrumentation Tests are what you’d write in the androidTest directory of your app. What separates an instrumentation test from a unit test is that instrumentation tests test components of your app working together instead of just their functionality as individual components.

Maybe you’ve heard of the Espresso testing framework at some point and if you’re like I was 5 minutes ago, you’d just be thinking what the hell is that? I’ll tell you. It’s a testing framework for Android. It’s integrated into the Android Support Library and thus, automatically setup in most Android projects. It’ll come into play when you build your test apk later.

Now what does Test Lab have to do with this? Well, Test Lab allows you to remotely run these tests en masse on devices of all sorts of models and configurations, courtesy of the devices hosted in that little Google Data Centre of theirs.

That’s all the theory out of the way. Let’s get to how to do it.

 

Configuring your gCloud Environment

First, download the Google Cloud SDK or make sure your installation is up-to-date.

gcloud components update

Login

gcloud auth login

And set your current project in gCloud

gcloud config set project PROJECT_ID

 

Configuring Tests for your app

Build your app to generate app-debug.apk and app-debug-test.apk. I found out this is a great tutorial for generating the test apk if you don’t know how to do it yet.

Next, you’ll be selecting from a list of device models, android versions, and locales to configure your test with.

gcloud firebase test android models list

Then run this command to get a list of available Android devices from the test lab.

If you want to get some specs for any of these models, you can describe any of them with a command like this.

gcloud firebase test android models describe Nexus5

Now do the same to get a list of Android OS versions:

gcloud firebase test android versions list

And locales

gcloud firebase test android locales list

All you need to do at this stage is take note of the IDs of the devices, versions, and locales to enter into the test command later.

 

Running the Instrumentation Test

gcloud firebase test android run 
      --type instrumentation 
      --app app-debug.apk 
      --test app-debug-test.apk 
      --device model=Nexus6,version=21,locale=en,orientation=portrait  
      --device model=Nexus7,version=19,locale=fr,orientation=landscape

Here it is. Boom. Put all the info and resources you’ve gathered earlier and run the instrumentation test.

After a few minutes, you’ll get the results appearing like this, as well as a link to analyse the test results in more detail. This is an example taken from the official docs.

Running Robo Tests with gCloud

You can also run Robo Tests in a very similar way. All you need to is change –type to robo instead of instrumentation, and add another field –timeout 90s or however long you want the timeout to be.

Subscribe to the Newsletter