High energy consumption is a challenging issue that an ever increasing number of mobile applications face today. However, energy consumption is being tested in an ad hoc way, despite being an important non-functional requirement of an application. Such limitation becomes particularly disconcerting during software testing: on the one hand, developers do not really know how to measure energy; on the other hand, there is no knowledge as to what is the energy overhead imposed by the testing framework.

In this paper, we design a methodology to measure the energy consumption of eight popular UI testing frameworks within six typical interactions and three helper methods. We have discovered that there are automation frameworks that increase energy consumption up to roughly 2200%. While limited in the interactions one can do, Espresso is observed to be the most energy-efficient framework. However, depending on the needs of the tester, Appium, Monkeyrunner, or UIAutomator are good alternatives.

In practice, results show that deciding which is the most suitable framework is vital. We provide a decision tree to help developers make an educated decision on which framework suits best their testing needs. Finally, we show that helper methods to find components in the interface – i.e., lookup methods – should be minimized to prevent affecting energy results. In particular, lookup methods based on the content of the UI component need to be avoided. They consistently yield poor energy efficiency when compared to lookups based on id (e.g., in Espresso it creates an overhead of 600%).