Monday, September 19, 2011

Android Application UI Testing (with monkey and monkeyrunner)

Android has some built in UI testing tools. These tools can be used for automated UI testing. However the tools are not so simple to use. This post is an attempt to set a guideline towards using these tools.

There are 2 main UI testing tools available

1.monkey (aka. UI/Application Exerciser Monkey)

This tool is a command line based tool that can be primarily used to stress test your application UI. It is the simplest tool to use. Here's how..

Running monkey as an application UI stress tester (runs a random set of commands on the application. useful to UI stress testing)

- Open a command console.
- First direct your console to the location of the adb (Android Debug Bridge) program in the android sdk. Typically you can use the following command for this..

$cd path_to_android_sdk/platform-tools/

path_to_android_sdk should be the path to the sdk on your pc

- Now make sure you device is connected with the application running on it or that the emulator is running the application.

- To test on device issue the following command in the console

$./adb -d shell monkey -p package_name -v 1000

(replace -d with -e to test on the emulator.
package_name is the name of the application package abnd it usually begins with com.
-v specifies the number of UI events to be generated, in this case we ask it to generate 1000 random events)

Now you will see the monkey stress testing your app.

If you get force close message while running monkey you have just discovered a bug that needs fixing. You can run

./adb logcat

in the console to generate the relevant logs.


Running specific commands in monkey
The monkey tool can also be used to run a specific set of commands on the application. However it is easier to use the monkeyrunner for this purpose.

On the connected device (to run on emulator simply replace -d with -e)

./adb -d shell monkey -p package_name --port 1080 &
./adb -d forward tcp:1080 tcp:1080
telnet localhost 1080

Now the following will be printed on cmd line..

Trying ::1...
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

Now you can type in your instructions

>tap 150 200

you can write all instruction into a script (script.txt) as below..

# monkey
tap 100 180
type 123
tap 100 280
press DEL
press DEL
press DEL
press DEL
press DEL
press DEL
press DEL
press DEL
type -460.3

now run..

./adb -d shell monkey -p package_name --port 1080 &
./adb -d forward tcp:1080 tcp:1080
nc localhost 1080 < script.txt


2. monkeyrunner

The monkeyrunner tool is an API for writing automated UI tests. You can use it to write specific scripts that run a series of commands and inspects the output by taking screenshots etc. The android SDK includes two special scripts written using monkeyrunner which help in running automated UI tests. The scripts are..

monkey_recorder.py
monkey_playback.py

Copy these scripts to /tools folder in the android sdk. Set the console path to the tools directory. Open up the application on the emulator.

You can run the recorder as follows..

./monkeyrunner monkey_recorder.py

Below is a screenshot of monkey_recorder recording actions for calculator.



Use 'Export Actions' button to export the set of actions into a script (eg: calc.mr)

now run the cal.mr script on a connected device as follows.. (first make sure the emulator is shut down)

./monkeyrunner monkey_playback.py calc.mr

For playback to run properly you need to take precautions of setting the correct wait times and using the correct press, fling, type methods in the recorder.

Conclusion: Android has a good collection of tools meant to enable UI test automation. However they still have a lot more room for improvement especially in terms of ease of use and efficiency.

References:

http://developer.android.com/guide/developing/tools/monkey.html
http://developer.android.com/guide/developing/tools/monkeyrunner_concepts.html
Android Application Testing Guide - Diego Torres Milano

40 comments:

  1. Hello Harini,
    Do we need to make any changes in the monkey_recorder.py file, Because when i run the command ./monkeyrunner monkey_recorder.py I'm getting a blank screen.

    ReplyDelete
  2. by blank screen do you mean the monkey recorder application opens but doesn't display the android screen inside it? this happens when the emulator is not running. recording doesn't work for a connected device so you need to have the android emulator running.

    ReplyDelete
  3. Hi Harini,
    Yes, the application opens but doesn't display the android screen. And I am running the emulator. Even though its the same blank screen.

    Thanks,
    GS

    ReplyDelete
  4. if your device is also connected at the time you are running the emulator monkeyrunner may be detecting your device instead of your emulator. monkey_recorder.py cannot read and display the screen of your device. disconnecting the device and having the emulator run should work.

    ReplyDelete
  5. Hello Harini,
    I did not connect any device to my PC. I am just running the emulator nothing other than that. Sorry to ask, can i have your personal mail id. or can you mail me to gerpatis.raghava@gmail.com

    Thanks,
    GS

    ReplyDelete
  6. Hello Harini,
    I am able to run monkeyrunner_recorder.py and know how to record but i am unable to get any output of monkeyrunner_playback.py script.mr file
    Please Suggest!
    Thanks
    MR

    ReplyDelete
  7. I like your blog technique.This is one of the great post.I like your blog out standing performance.By read your post I came to know Android has to Build some UI tools.This is one of the specific post.
    Android app developers

    ReplyDelete
  8. hi,

    the recorder and playback files are not available to download. Can you please let me know regarding writing scripts for monkey and how to run it on device? or any source for tht

    Thanks

    ReplyDelete
  9. Sorry, didn't notice the dead links. Just updated them..

    ReplyDelete
  10. Hi Harini,

    Shall we run the monkey tool at a time on two command prompts????

    I mean type and run monkey commands one after the another on command prompt immediatly???

    which command prompt events are generated and applied to emulator???

    Can u plz clarify dis??

    Thanks in Advance,,
    Anji

    ReplyDelete
  11. Hi Harini
    Can you please explain how these scripts are directly in the monkey exerciser.

    Thanks in advance
    Teresa

    ReplyDelete
  12. @Anji

    sorry i'm not clear with what you are asking. can you explain what you are trying to do?

    @Teresa

    the monnkeyrunner program is in the tools folder of android sdk. Download the monkey_recorder.py and monkey_playback.py scripts above and copy them into the same tools folder where monkeyrunner is. Then you can change to the tools directory in terminal and call the following commands ..

    ./monkeyrunner monkey_recorder.py
    ./monkeyrunner monkey_playback.py script_name.mr

    if you keep the scripts in a separate folder you can call them using the full file path name.

    ReplyDelete
    Replies
    1. I also like to know how to use the following in a windows environment

      ./adb -d shell monkey -p package_name --port 1080 &
      ./adb -d forward tcp:1080 tcp:1080
      nc localhost 1080 < script.txt

      Running the first line in a command prompt
      adb -d shell monkey -p package_name --port 1080
      is showing an error "Error binding to network socket."

      Delete
    2. Working Nice , THX , Actually the Rule was those APK which was having the Launcher Only , It Works , If we Integraed many apk with one Common APK means , It wont Work "The Main Apk (com ) only Works

      Delete
    3. I am running into same error as Sam mentioned above, Wondering how did you resolve the issue?

      Delete
  13. Nice article.. helped me to get started...

    ReplyDelete
  14. I tried by following the steps, but it never works, my PC is not able to telnet to my device, any idea?

    BTW,
    If my device and PC are under the same WiFi access point, it should be possible to telnet from my PC to the monkey on my device through the network without having to set port forwarding?

    ReplyDelete
  15. hii harini i was able to record and i was unable to view that one canu plz provide the exact command for that and one more i need some resources for writing python scripts for monkey runner .............i was unable to find any where in th surfing

    ReplyDelete
    Replies
    1. You need to reset the application position on the emulator/device to the point where you started the recording, before calling the ./monkeyrunner monkey_playback.py "script_name" command. This way you can playback the recorded commands from the beginning.

      Delete
  16. Hi Harini,

    Can you explain, how to reboot offline android phone using monkey tool or monkeyrunner tool ?

    Thanks & Regards,

    ReplyDelete
    Replies
    1. Reboot is not possible using adb, monkey or monkeyrunner.

      Delete
    2. you can use this command to reboot your device, it worked for me.

      device.shell('reboot')

      Regards
      Nik

      Delete
  17. Awesome blog really helpful lot for me ..
    harini rocks....

    ReplyDelete
  18. Thanks of your posting, which made me to join Testing tools online training @ www.Monstercourses.com

    ReplyDelete
  19. Thanks for sharing knowledge. very help full me and save lot of time.

    ReplyDelete
  20. Your articles make complete sense out of each topic.
    super one click root

    ReplyDelete
  21. Info is out of this world, I would bang to see more from your writers.
    check this link right here now

    ReplyDelete
  22. Guys for rebooting your device please use below command
    it worked for me.All the best.

    device.shell('reboot')

    Nikhil

    ReplyDelete
  23. Hi,

    I followed steps for using monkey.

    ./adb -d shell monkey -p package_name --port 1080 &
    ./adb -d forward tcp:1080 tcp:1080
    telnet localhost 1080

    It prints:

    Trying ::1...
    Trying 127.0.0.1...
    Connected to localhost.
    Escape character is '^]'.
    Connection closed by foreign host.

    Wats the reason for 'Connection closed by foreign host'?

    ReplyDelete
  24. Is there any reason why the press buttons don't work? Anytime I try the menu or home press buttons they don't work. Also is there any way to extend a duration of a touch? I would like to test taking pictures but the touch input is far too fast for the camera button. Thanks

    ReplyDelete
  25. Hii, how to get the time in monkey logs?

    ReplyDelete
  26. I run monkeyrunner monkey_recorder.py ok.
    Now i want to see show visual feedback for touches. My mean when i touch on emulator i will see feedback for touches on real device. I checked into "show touches" and "pointer location" on real device. NEED to help.
    Thanks

    ReplyDelete
  27. Interesting information I haven’t been through such information in a long time
    free Samsung Galaxy S4 root

    ReplyDelete
  28. Hi Harini,

    When am executing the monkey recorder script with the below mentioned command am getting error.

    In Windows: monkeyrunner.bat monkey_recorder.py

    C:\Users\Murali\Desktop\Murali\AndroidADT\tools>monkeyrunner.bat monkey_recorder
    .py
    131111 15:06:07.463:S [main] [com.android.chimpchat.adb.AdbChimpDevice] Timeout
    while trying to create chimp mananger
    131111 15:06:07.473:S [main] [com.android.monkeyrunner.MonkeyRunnerOptions] Scri
    pt terminated due to an exception
    131111 15:06:07.473:S [main] [com.android.monkeyrunner.MonkeyRunnerOptions]Trace
    back (most recent call last):
    File "C:\Users\Murali\Desktop\Murali\AndroidADT\tools\monkey_recorder.py", lin
    e 19, in
    device = mr.waitForConnection()
    at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:
    191)
    at com.android.chimpchat.adb.AdbChimpDevice.(AdbChimpDevice.java:7
    3)
    at com.android.chimpchat.adb.AdbBackend.waitForConnection(AdbBackend.jav
    a:122)
    at com.android.chimpchat.ChimpChat.waitForConnection(ChimpChat.java:91)
    at com.android.monkeyrunner.MonkeyRunner.waitForConnection(MonkeyRunner.
    java:75)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)

    java.lang.NullPointerException: java.lang.NullPointerException

    131111 15:06:07.473:S [main] [com.android.monkeyrunner.MonkeyRunnerOptions]
    at org.python.core.Py.JavaError(Py.java:495)
    131111 15:06:07.473:S [main] [com.android.monkeyrunner.MonkeyRunnerOptions]
    at org.python.core.Py.JavaError(Py.java:488)
    131111 15:06:07.473:S [main] [com.android.monkeyrunner.MonkeyRunnerOptions]
    at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:188)
    131111 15:06:07.473:S [main] [com.android.monkeyrunner.MonkeyRunnerOptions]
    at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:204)
    131111 15:06:07.473:S [main] [com.android.monkeyrunner.MonkeyRunnerOptions]
    at org.python.core.PyObject.__call__(PyObject.java:371)
    131111 15:06:07.473:S [main] [com.android.monkeyrunner.MonkeyRunnerOptions]
    at org.python.core.PyObject.__call__(PyObject.java:375)
    131111 15:06:07.473:S [main] [com.android.monkeyrunner.MonkeyRunnerOptions]
    at org.python.pycode._pyx0.f$0(C:\Users\Murali\Desktop\Murali\AndroidADT\tools\m
    onkey_recorder.py:20)
    131111 15:06:07.473:S [main] [com.android.monkeyrunner.MonkeyRunnerOptions]
    at org.python.pycode._pyx0.call_function(C:\Users\Murali\Desktop\Murali\AndroidA
    DT\tools\monkey_recorder.py)
    131111 15:06:07.473:S [main] [com.android.monkeyrunner.MonkeyRunnerOptions]
    at org.python.core.PyTableCode.call(PyTableCode.java:165)
    131111 15:06:07.473:S [main] [com.android.monkeyrunner.MonkeyRunnerOptions]
    at org.python.core.PyCode.call(PyCode.java:18)

    ReplyDelete
    Replies
    1. you need to merely run as monkeyrunner monkey_recorder.py
      and not as monkeyrunner.bat monkey_recorder.py
      if still it doesn't work, then use ./monkeyrunner monkey_recorder.py as we use in Linux/Unix based OS.

      Good Luck...!!!

      Delete
  29. Great post, thank you very much, please write more and more about this. Very interesting and understand.Fell in love with you.

    ReplyDelete
  30. Hi Harini,
    I am learning Monkeyrunner and i need some clarification regarding this tool.
    For any particular monkeyrunner script how can i decide whether test is Pass or Fail.
    Thanks

    ReplyDelete
  31. A very helpful post. Thanks for sharing. Cheers!

    ReplyDelete
  32. Thank you for posting this informative post. It is really useful.

    ReplyDelete