Providing Feedback and UI Context to Programmers Writing Web Automation Scripts
To more efficiently and effectively perform frequent, tedious, or inaccessible web tasks, web users can leverage web automation macros to programmatically click, type, and perform other page operations. Developers commonly use automation libraries such as Selenium to write custom web automation scripts, but this comes with challenges. A developer must identify robust navigation and element selection logic that works across a variety of inputs and website pages. In particular it can be challenging to choose meaningful and robust CSS selectors that query the appropriate user interface (UI) elements from the DOM. We believe developers need better tools for understanding the effects their script has on a website. We built a prototype web automation IDE that embeds UI snapshots, provides live feedback on CSS selectors, and presents these across different scenarios so the programmer can assess script robustness. We will demo our prototype to get feedback and to discuss the challenges and opportunities of UI automation.