CAIN 2022
Mon 16 - Tue 17 May 2022
co-located with ICSE 2022
Mon 16 May 2022 06:45 - 07:00 at CAIN main room - Quality Assurance Chair(s): Henry Muccini

There is a growing interest in industry and academia in machine learning (ML) testing. We believe that industry and academia need to learn together to produce rigorous and relevant knowledge. In this study, we initiate a collaboration between stakeholders from one case company, one research institute, and one university. To establish a common view of the problem domain, we applied an interactive rapid review of the state of the art. Four researchers from Lund University and RISE Research Institutes and four practitioners from Axis Communications reviewed a set of 180 primary studies on ML testing. We developed a taxonomy for the communication around ML testing challenges and results and identified a list of 12 review questions relevant for Axis Communications. The three most important questions (data testing, metrics for assessment, and test generation) were mapped to the literature, and an in-depth analysis of the 35 primary studies matching the most important question (data testing) was made. A final set of the five best matches were analysed and we reflect on the criteria for applicability and relevance for the industry. The taxonomies are helpful for communication but not final. Furthermore, there was no perfect match to the case company’s investigated review question (data testing). However, we extracted relevant approaches from the five studies on a conceptual level to support later context-specific improvements. We found the interactive rapid review approach useful for triggering and aligning communication between the different stakeholders.

Mon 16 May

Displayed time zone: Eastern Time (US & Canada) change

06:30 - 07:30
Quality AssuranceCAIN 2022 at CAIN main room
Chair(s): Henry Muccini University of L'Aquila, Italy
06:30
15m
Research paper
What is Software Quality for AI Engineers? Towards a Thinning of the FogResearch Paper
CAIN 2022
Valentina Golendukhina University of Innsbruck, Valentina Lenarduzzi University of Oulu, Michael Felderer University of Innsbruck
06:45
15m
Research paper
Exploring ML testing in practice - Lessons learned from an interactive rapid review with Axis CommunicationsResearch Paper
CAIN 2022
Qunying Song Lund University, Markus Borg RISE Research Institutes of Sweden, Emelie Engstrom Lund University, Håkan Ardö Axis Communications, Lund, Sweden, Sergio Rico Lund University, Sweden
Pre-print
07:00
15m
Research paper
Quality Assurance of Generative Dialog Models in an Evolving Conversational Agent Used for Swedish Language PracticeResearch Paper
CAIN 2022
Markus Borg RISE Research Institutes of Sweden, Johan Bengtsson Lund University, Harald Österling Lund University, Alexander Hagelborn NordAxon AB, Isabella Gagner NordAxon AB, Piotr Tomaszewski RISE Research Institutes of Sweden
07:15
15m
Other
Discussion on Quality Assurance
CAIN 2022


Information for Participants
Mon 16 May 2022 06:30 - 07:30 at CAIN main room - Quality Assurance Chair(s): Henry Muccini
Info for room CAIN main room:

Click here to go to the room on Midspace