Using GUI Ripping for Automated Testing of Android Apps

1. Using GUI Ripping for Automated Testing of Android Apps Domenico Amalfitano Salvatore De Carmine Anna Rita Fasolino Porfirio Tramontana Dipartimento di Informatica e Sistemistica University of Naples Federico II, Italy AtifMemon Department of Computer Science University of Maryland

2. Motivation • Android is a very popular operative system and framework for mobile devices • Its diffusion is increasing: recently it overtook ioS and Symbian in terms of diffusion • Over than 500,000 apps are now on the Google Play market • The most part of the apps are realized by single programmers or little teams and lacks in quality • Android apps are often developed according to an evolutionary life cycle, with very frequent releases via Google Play • Bugs causing failures and, in particular, crashes are very common • There is a strong need for testing automation processes and tools • In this talk, our techniques and tool supporting a process of Android GUI Testing Automation will be presented

3. Androidapps • An Android application is composed of several types of Java components instantiated at run-time: • Activities, Services, Intents, Broadcast Receivers and Content Providers. • An Activitycomponent: • is responsible for presenting a visual user interface; • provides a screen with which the users can interact in order to do something. • An Android application usually includes one or more Activity classes that extend the base Activity class provided by the Android development framework. • Android Applications are Event-Driven Systems • Three types of events can be triggered: • user events (such as Click, MouseOver, etc.); • events related to the lifecycle of the Activity classes • onStart, on Pause, onResume, onStop, onDestroy, … • events that are triggered by other input sources • such as GPS receiver, phone, network, etc.

4. Existingapproaches to AndroidTesting Automation • JUnit Testing • Android Testing Framework • Supports the coding of executable test classes • Classes supporting the instrumentation and monitoring of Android apps are provided in the Android framework itself • Libraries such as Robotiumand Monkeyrunner extends the framework • Random Testing • Monkey (included in the Android framework) supports the randomly automatic exploration of Android apps • Model Based Testing • TEMA Tools (Takala, Katara, Harty, 2011) represent a first tentative of approach for the generation of test code from a design model of Android app

5. Android GUI Ripping • We propose a technique and a tool for: • Automatic exploration of the GUI of an Android application via a systematic Ripping approach • Similar to approachesadopted for Web crawling • Crash testing • GUI model reverse engineering • JUnit test case generation

6. Android GUI RippingAlgorithm • Customizable • Selection of types of events to be fired • Adoption of a GUI traversing strategy (e.g. depth first, breadth first, etc.) • Termination criteria used to stop the GUI traversal • Setting of specificpreconditions • Efficient • Provides heuristic solutions to the well known problem of GUI interface states and events explosion • GUI exploration criteria based on a Interface Equivalence Criterion

7. RippingAlgorithm ≡ ≡ Rotate Click Pages Click New Post Press Menu Click Refresh … Click About Click Save Click Add Account Crash … Click Edit … … Task List Initialization; while (Task List Is Not Empty) { Extract a Task From The Task List; Execute the Task; Abstract the Current GUI Abstract State; Update the GUI Tree Model; if (GUI Exploration Criterion) then { Define New Tasks ; Add New Tasks To The Task List; } } Task List Initialization; while (Task List Is Not Empty) { Extract a Task From The Task List; Execute the Task; Abstract the Current GUI Abstract State; Update the GUI Tree Model; if (GUI Exploration Criterion) then { Define New Tasks ; Add New Tasks To The Task List; } }

8. Crash TestingProcess • Preconconditions are set by generating a Snapshot Image of the device for the emulator • In the task execution step, the GUI Ripping algorithm is executed

9. A Demonstrationalexample • Measured metrics • Effectiveness • Number of crashes found • Number of different bugs causing crashes • LOCs coverage • Efficiency • Number of test cases • Number of events • Ripping time • Android for Wordpress • A Wordpress client for Android • 334 classes, ~10,000 LOCs • More than 250,000 downloads • A set of three preconditions • No Login • Single Blog • Multiple Blogs • Comparison between Ripping and Random Testing with Monkey

10. Results

11. Bugs and Crashes • The 14 crashes found by the Ripping process belongs to four different bugs • Bugs have been submitted to the bug tracking system and accepted/fixed by the programmer • Monkey was able to find only one of these bugs

12. Conclusions and Future Works • Our technique supports the execution of completely automatic generation and execution of test cases for Android applications. In our preliminary demonstration example: • Real bugs and crashes have been detected; • Good level of code coverage has been reached; • Testing time is compatible with smoke testing processes; • Reusable test cases has been automatically generated; • Our technique demonstrates its effectiveness with respect to standard Random testing tools such as Monkey

13. Future Works • This presentation covers the first part of a larger project on testing automation of Android applications • Some future works • Publication of the tool • As a standalone tool • As a service • A preliminary demo is available at http://wpage.unina.it/ptramont/GUIRipperWiki.htm • Further experiments with more applications and ripping parameters • Delay Time, ripping strategy (depth first/breadth first, activity equivalence criteria, maximum depth), input generation strategies, … • Sensor event testing

14. Time is over ... Is there any question?

18. Output

19. Android GUI Conceptual Model • An Interface is a sort of instance of a user-defined class extending Activity • Similar interfaces are grouped together • Events can be • User events, related to the interaction of the user with the GUI • System events, related to external sources or to the lifecycle of the activities

20. An example of FSM

21. Otherexperimentalresults