Android UI Testing at STRV

As STRV takes on more and more complex projects, there is an increasing need to create and automatically perform tests at each step of the development process, including functionality tests, integration tests, system integration tests and user acceptance tests.

Here, we will take a closer look at UI tests, which are part of functionality tests and ensure that the app is of high quality before it is released on the market. These tests help minimize subsequent errors and reduce future costs that might arise during deployment.

The basic idea behind UI testing is to test an app at a user level and verify that the app works correctly. You can test the app manually, but this type of test is time consuming, tedious and error-prone. The automated approach allows you to run your tests quickly and reliably in a repeatable manner.

Popular UI testing frameworks

Now let’s check out some of the most frequently used frameworks for UI testing.

Espresso is supported by Google. The core API is small, predictable and easy to learn and can be customized. Espresso tests leave your waits, syncs, sleeps and polls behind and manipulate and assert the app’s UI thread when it is at rest. Tests are compiled with your app, optimized for Android and run smoothly and fast. Espresso has been supported since API level 8 - Froyo.

Robotium is an open-source framework developed for writing automatic gray box testing cases. It can be used for testing Android apps where the source code is available as well as apps where only the APK file is available and the implementation details are unknown. Robotium is similar to Selenium but geared toward Android. It has support for Android features, including activities, toasts, menus and context menus. Robotium has been supported since API level 8.

Selendroid is a test automation framework for Android native, hybrid and mobile web apps. Tests are written using the Selenium 2 client API and support full integration as a node in a cloud service for scaling and parallel testing. Selendroid has been supported since API 10.

With Appium you don’t need to include an SDK or recompile the app for testing. Appium also allows you to use your own preferred test practices, tools and frameworks. These features allow Appium to provide true cross-platform automation.

Calabash is a free open-source project, developed and maintained by Xamarin, that provides a bridge for Cucumber tests to run and validate iOS and Android native and hybrid apps. It allows test code to interact with the apps, wherein each of these interactions consists of a number of end-user actions like gestures, assertions and screenshots.

####STRV’s experience with Espresso and Robotium

We have worked with Robotium 5.3.1 and Espresso 2.0 at STRV.

Robotium is a robust, well-documented and up-to-date library for UI testing. Writing tests is straightforward, and the learning curve is short. But it is necessary to insert a time-out in the code to make sure tests are not flaky on slow devices or specific Samsung devices.

"Flaky tests are, generally speaking, tests that fail intermittently. This can happen if there is an underlying bug that only happens intermittently. This could also happen if a test is timing-dependent and affected by environmental conditions (e.g. the speed of the machine running the test)."

After initial setup problems and importing static libraries, Espresso has become very helpful for writing complex tests. For instance, it allows you to write custom Hamcrest ViewHolder matchers through which you can find an item in RecyclerView based on the item’s content.

Here’s an example of the RecyclerViewActions.scrollToHolder method which demonstrates a deeper integration:

onView(withId(R.id.recycler_view)) 
.check(matches(isDisplayed()))
.perform(RecyclerViewActions.scrollToHolder(
new CustomViewHolderMatcher(hasDescendant(withText(“item text”)))))));
 
private static class CustomViewHolderMatcher 
extends TypeSafeMatcher <RecyclerView.ViewHolder> 
{ 
    private Matcher<View> itemMatcher = any(View.class); 

    public CustomViewHolderMatcher() { } public 

    CustomViewHolderMatcher(Matcher<View> itemMatcher) 
    {
        this.itemMatcher = itemMatcher; 
    } 

    @Override public boolean matchesSafely(RecyclerView.ViewHolder viewHolder) 
    { 
        return TrackViewHolder.class.isAssignableFrom(
        viewHolder.getClass()) && itemMatcher.matches(viewHolder.itemView); 
    }
}
//TrackViewHolder is a custom view holder class that RecyclerView uses in the adapter.

Another great feature is that Espresso waits for UI operations and a default AsyncTask in the thread pool. You can implement IdlingResource to handle all background operations running in IntentService.

Conclusion

Each version of Robotium adds new features and capabilities to UI testing. For instance, the RecyclerView method’s scrollRecyclerViewToBottom (int position) allows you to search for an item based on the item’s position, instead of its content. Robotium is a great tool for smaller apps, which allows them to prove their concept.

Espresso, meanwhile, is more flexible and allows you to write your own Hamcrest matcher, which generally works much better with RecyclerView, DrawerLayout, ActionBar and Pickers. You can also write more accurate and deeper integrated tests with Android. It's definitely worth the extra effort.