Automated Testing of Applications on Android using ROBOTIUM

Last two weeks have been lot of fun working on evaluating various automation tools for automated testing of applications on Android. I started with the primary purpose to calibrate a set automation tools to perform data driven automated testing applications on Android.

Start with setting up the environment using following tools:

Install JDK, Eclipse 3.4 (Ganymede) or greater and Android Development Tools plug-in.

Next install Android Emulator from the eclipse window menu select Android SDK and AVD Manager. Selecting the device and clicking on “Start” launched the emulator device with home page of the device.

After the environment was setup successfully, I started with creating a very simple temperature convertorapplication to better understand the architecture & UI interfaces of the Android SDK.(source:http://www.vogella.de/articles/Android/article.html

Now the real fun of testing the temperature convertor application started using the Robotium framework. I started by downloading the Robotium solo.jar from (http://code.google.com/p/robotium/downloads/list) and deploy the APK (Android Package File. This is a variant of the JAR file format and is used for the distribution and installation of bundled components onto the Android mobile device platform) of the application to test on the emulator. There is no dependency on source code of the application under test, however we need to know package name and start activity details. This is the pre-requisite to start test script development.

I have used ADB tools provided along with SDK tool kit to identify the package name and start activity details. After finding these details, I created a test project, added Robotium.jar file and modified the test project in AndroidManifest.xml file with package name and added a class file.We need to modify the script with the package and start activity details to get started in using the solo object to perform action on the UI elements.This approach best suits for the real time implementations.

So my base was up and running to test the applications on the target system successfully. However, the automation engineer in me pushed me further to extend the tests by making them data driven. So I thought of using Apache POI for Excel to perform data driven testing. However, after some analysis, I found that Apache POI for Excel handling is not good to use with Android Test as the POI jar file has to be integrated with test builds, thus increasing the size of APK and build deployment time. I was also uncertain if data accessing using POI API is supported by Android device or not.

I thought of exploring the feasibility of other approaches like accessing data from XML files OR using a database system for test data. After some research, I zeroed on using SQLite, Android built-in database as it seemed apt for implementing Data Driven testing.

Followed the approach outlined below for creating data driven scripts using SQLite:

1. Creating SQLite database

2. Copy .sqlite DB to Assets folder.

3. Configure DB details in Strings.XML file.

4. Create a class for database handler

5. Created instrumentation andretrieved Context handler for Current Test project in Main test Class.

6. Executed Query and opened a Cursor

7. Readdata from DB using Cursor

I executed the application deployed on the target system using data driven approach defined above– and Hey, I was successful in executing the data driven test on the application deployed on the target system – nice!

The next step was to generate a user friendly test execution reports potentially in HTML format. In the case of mobile devices, script execution happens on target system which could be a real mobile device or device simulator and the results get generated on the target system itself. We don’t have any control on host machine file system from target system. So, I developed a HTML Report generation class in java and used file system API to generate automated HTML report. This HTML report API do not requires any configuration. All we need to do is to plug-in the jar and make a call to method to report some status.

This approach generates HTML report on target system and I used the ADB tool to pull back the result to Host System and was able to achieve it using batch file running in host system.

With this accomplished, I will be looking at few other things before I can start putting together a hybrid, data driven framework for testing application for android OS.

. Ability to capture screenshots using either a java plug-in or python script in test script

. Ability to automate UI elements if objects are not identified by visible text or index

. Implementing Auto Mail feature to get HTML results as attachments

Some of the challenges faced while trying to automate are outlined below.

Issues & Solutions

Issue#1: Error “Failure [INSTALL_PARSE_FAILED_NO_CERTIFICATES]” while deploying APK file

Reason:If the APK file doesn’t contain a signature, then deploying is not possible.

Solution:Assign a signature using either debug key or private key signature using reassign.jar and jarsigner tools

Issue#2:Unable to implement Automation solution for publically available applications/apps developed on different PCs

Reason:Signature of Public application is not matching with signature of Test Project APK file

Solution:Reassign the Test Project signature to Public application using reassign.jar and jarsigner tools

Issue#3:Unable to access the Configdata(DB Details) stored in Strings.xml using the Context object,

Reason:By default context object gets created for the base application, not for the test project instance.

Solution:create a context for test project instance using getInstrumentation(

Issue#4:Getting Results from target system to Host System by executingadb pull command using JAVA getRuntime() throws exception

Reason:ADB pull command expectssource & target files locations. This command is made to access target system shell from host system.

Solution:Develop a batch to use adb pull command & execute this file manually

I would encourage others to add to this list of issues faced during environment setup, application deployment or automation so that I can increase my knowledge and we can share some our ideas on how to build a robust data driven framework for automated testing of application on Android