Software testing is vital, yet expensive and time-consuming. This essential part of the software development process includes testers performing many repeated actions in test execution and management. Use of automation and tools could reduce costs and timescale, while providing consistency by removing human error during repetitive activities. Challenges for successful tools and automation adoption have been identified both in academic research and in industry practice, including technical, managerial, skills-related and usability issues. We set out to investigate what usability improvements would aid successful tool adoption, and discovered that usability, while a necessary attribute, is not sufficient to ensure success, and the belief in usability as a sufficient cure for automation shelfware might be an illusory phenomenon which disguises potential difficulties when using tools longer term. This illusion of usability includes a belief that UI attractiveness is sufficient for tool usability, a belief that testers come from a narrow group of personas, and a belief that skill levels and requirements for tools are static. This may lead to frustration for testers, and therefore reluctance to use tools and automation. We summarise our findings and outline our proposed next research steps.