An automated model-based approach to repair test suites of evolving web applications
Context: Web applications evolve frequently to incorporate new functionality and adopt the recent trends, but even minor adjustments can result in major changes to the web page’s underlying structure. Capture-Replay tools are widely used for the automated testing of web applications. The scripts written using these Capture-Replay tools are strongly coupled with the web elements of web applications. Even simple changes, such as slight modifications in a web page layout may break the existing test scripts, because the web elements that the scripts are referencing may not be valid in the new version. Objective: In this work, we propose a model-based automated approach to repair the broken test scripts of Capture-Replay testing tools. This approach covers the various types of changes of web elements that may result in the breakage of test scripts. No other existing work provides such a comprehensive test repair strategy and are testing framework dependent. Methodology: This model-based web test repair approach is independent of the underlying Capture-Replay tool. To provide a tool independent methodology, we develop a UML profile that allows the capture of various concepts related to Capture-Replay test scripts for web applications. For this purpose, we extend the UML Test Profile (UTP) and add important concepts relevant to capture and replay test scripts of web applications. It uses a DOM-based strategy that automatically detects the DOM-level differences between the two versions of the web application. Based on the identified differences, test suite is classified as reusable, obsolete, retestable, and then fixes them by applying the repair heuristics. We also develop an open-source tool that automates the proposed test repair approach. The proposed approach is evaluated by conducting a series of four evaluations on one industrial and six open-source case studies to evaluate the effectiveness of our proposed approach in terms of repairing the broken test scripts, coverage of DOM elements, and fault-finding capability. We also evaluate the usefulness of the repaired test scripts according to the opinion of professional testers. We also compare our approach with the only other available DOM-based test repair approach, WATER. Results: The results of the empirical evaluation indicate that the proposed approach the proposed approach effectively repairs the 91% of broken web test scripts and achieves a similar DOM-coverage, as by the original test suite, on the evolved versions of subject applications. A team of professional testers found the suggested repairs, for different types of test breakages, useful for the regression testing of evolving web applications. Furthermore, the DOM-based fault-finding capability of the repaired test suite is equivalent to the original test suite. Our empirical evaluation on 528 Selenium web driver test scripts of seven web applications shows that the proposed can effectively repair 83% of the overall breakages, whereas the existing technique WATER repairs 58% test breakages which only includes attribute-based locators and broken assertion values. Conclusion: We have proposed a scalable and automated strategy to migrate the existing automated test scripts (e.g., Capture Replay) towards the new and evolved version of the web application. Our approach repairs the test scripts that may be broken due to the breakages (e.g., broken locators, missing web elements) reported in the existing test breakage taxonomy. Our approach is based on a DOM-based strategy and is independent of the underlying Capture-Replay tool. We developed a tool to demonstrate the applicability of the approach. We perform an empirical study on seven subject applications. The results show that the approach successfully repairs the broken test scripts while maintaining the same DOM coverage and fault-finding capability.