WAIN'21
Sun 30 - Mon 31 May 2021
co-located with ICSE 2021
Mirror WAIN’21 page

ml-devops

Welcome to WAIN’21 - 1st Workshop on AI Engineering – Software Engineering for AI

The aim of the workshop is to bring together researchers and practitioners in software engineering, in data-science and AI, and to build up a community that will target the new challenges emerging in Software Engineering that AI/data-science engineers and software engineers are facing in development of AI-based systems. The workshop will be highly interactive: In addition to the invited keynotes and short paper presentations, there will be several discussion sessions. We plan to combine local and remote participation.

Keynote: Lionel Briand, Trustworthy Machine Learning-Enabled Systems, Sun May 30

Lionel Briand This talk will provide a personal perspective on the state of art regarding the automated testing and analysis of software systems enabled by machine learning. Such systems typically contain components relying on machine learning, whose behavior is not specified or coded but driven by training data, but which interact with other components in the system and play a critical role. Typical examples include cyber-physical systems that rely on machine learning in their perception (e.g., analyzing camera images) and control (e.g., sending commands to actuators) layers. In my reflections, I will rely both on my analysis of the state of the art and personal experience in research projects carried out with industrial partners in the automotive domain.

Lionel C. Briand is professor of software engineering and has shared appointments between (1) School of Electrical Engineering and Computer Science, University of Ottawa, Canada and (2) The SnT centre for Security, Reliability, and Trust, University of Luxembourg. He is the head of the SVV department at the SnT Centre and a Canada Research Chair in Intelligent Software Dependability and Compliance (Tier 1). He holds an ERC Advanced Grant, the most prestigious European individual research award, and has conducted applied research in collaboration with industry for more than 25 years, including projects in the automotive, aerospace, manufacturing, financial, and energy domains. He is a fellow of the IEEE and ACM. He was also granted the IEEE Computer Society Harlan Mills award (2012) and the IEEE Reliability Society Engineer-of-the-year award (2013) for his work on model-based verification and testing. More details can be found on: http://www.lbriand.info.

Keynote: Errol Koolmeister, Engineering AI at H&M group, Mon May 31

Errol Koolmeister Talk.H&M has invested heavily in AI the last few years and have gone from shattered decentralized projects into a large central effort focusing on amplifying all core operational decisions with AI. This effort requires them to rearchitect many of the core systems to support horizontal scaling. The keynote will go through the journey and share some of the key insights from this successful endeavor.

As head of AI Foundation for H&M group, Errol Koolmeister is currently working on setting up and overseeing the AI projects in the group. Prior to H&M, Errol worked as a Director of Data Science for ThinkBig Analytics, a Teradata company where he was responsible for setting up and delivering on AI projects across the Nordics, Eastern Europe and Russia. Prior to that he was in London as a Lead Data Scientist for the Vodafone group, but he originally started his career in Nordea bank where he spent about 10 years in various analytics roles.

Schedule Overview

Sunday May 30
13:00-130:15 Opening session
13:15 - 14:15 Keynote: Lionel Briand
14:15 - 14:30 Speed dating I
14:30 - 15:00 Virtual coffe break
15:00 - 16:00 Session 1: Challenges in developing Machine-Learning-Enabled Systems - Experience from the trenches.
16:00 - 16:15 Speed dating II
16:15 - 16:30 Virtual coffe break
16:30 - 17:30 Session 2: Engineering Trustworthy AI systems
17:30-18:00 Position papers presentations
Monday May 31
9:00-10:00 Session 3: Software engineering lifecycle phases for AI systems
10:00 - 10:15 Speed dating III
10:15 -11:15 Session 4: Applying ML technologies
11:15 - 11:30 Virtual coffe break
11:30 - 12:30 Panel 1
12:30 - 13:30 Lunch
13:30 - 14:30 Keynote Errol Koolmeister
14:30 - 14:45 Speed dating IV
14:45 - 15:00 Virtual coffe break
15:00 - 16:00 Session 5: Designing AI systems I
16:00 - 16:15 Speed dating V
16:15 - 17:00 Session 6: Designing AI systems II
17:00 - 17:45 Panel 2
17:45 - 18:00 Closing session
Dates
You're viewing the program in a time zone which is different from your device's time zone change time zone

Sun 30 May

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

13:00 - 13:15
Opening SessionWAIN'21 at WAIN Room
Chair(s): Jan Bosch Chalmers University of Technology, Ivica Crnkovic Chalmers University of Technology, Helena Holmström Olsson Malmö University, Lucy Ellen Lwakatare University of Helsinki, Finland
13:00
15m
Talk
Opening session
WAIN'21
Ivica Crnkovic Chalmers University of Technology, Jan Bosch Chalmers University of Technology, Helena Holmström Olsson Malmö University, Lucy Ellen Lwakatare University of Helsinki, Finland
Media Attached
13:15 - 14:15
WAIN'21 Keynote 1WAIN'21 at WAIN Room
Chair(s): Ivica Crnkovic Chalmers University of Technology
13:15
60m
Keynote
Lionel Briand, Trustworthy Machine Learning-Enabled Systems
WAIN'21
Lionel Briand University of Luxembourg and University of Ottawa
Media Attached
14:15 - 14:30
Speed dating IWAIN'21 at WAIN Room
14:30 - 15:00
Virtual Coffee Break IWAIN'21 at WAIN Room
15:00 - 16:00
Session 1: Challenges in developing Machine-Learning-Enabled Systems - Experience from the trenches.WAIN'21 at WAIN Room
Chair(s): Henry Muccini University of L'Aquila, Italy
15:00
20m
Paper
Characterizing and Detecting Mismatch in Machine-Learning-Enabled Systems
WAIN'21
Grace Lewis Carnegie Mellon Software Engineering Institute, Stephany Bellomo Software Engineering Institute, Ipek Ozkaya Carnegie Mellon Software Engineering Institute
Pre-print Media Attached
15:20
20m
Talk
Linnaeus: A highly reusable and adaptable ML based log classification pipeline
WAIN'21
Media Attached
15:40
10m
Paper
Towards Productizing AI/ML Models: An Industry Perspective from Data Scientists.
WAIN'21
Filippo Lanubile University of Bari, Fabio Calefato University of Bari, Luigi Quaranta University of Bari, Italy, Maddalena Amoruso , Fabio Fumarola , michele filannino
Pre-print Media Attached
15:50
10m
Paper
Who Needs MLOps: What Data Scientists Seek to Accomplish and How Can MLOps Help?
WAIN'21
Pre-print Media Attached
16:00 - 16:15
Speed dating IIWAIN'21 at WAIN Room
16:15 - 16:30
Virtual Coffee Break IIWAIN'21 at WAIN Room
16:30 - 17:30
Session 2: Engineering Trustworthy AI SystemsWAIN'21 at WAIN Room
Chair(s): Aneta Vulgarakis Ericsson, SE
16:30
20m
Full-paper
Robust Machine Learning in Critical Care - Software Engineering and Medical Perspectives
WAIN'21
Miroslaw Staron University of Gothenburg, Helena Odenstedt Herges
Pre-print Media Attached
16:50
20m
Full-paper
MLOps Challenges in Multi-Organization Setup: Experiences from Two Real-World Cases
WAIN'21
Pre-print Media Attached
17:10
10m
Short-paper
Towards Risk Modeling for Collaborative AI
WAIN'21
Matteo Camilli Free University of Bozen-Bolzano, Michael Felderer University of Innsbruck, Andrea Giusti , Anna Perini Fondazione Bruno Kessler, Barbara Russo Free University of Bolzano, Angelo Susi Fondazione Bruno Kessler
Pre-print Media Attached
17:20
10m
Short-paper
Practices for Engineering Trustworthy Machine Learning Applications
WAIN'21
Alex Serban Radboud University, Koen van der Blom , Holger Hoos , Joost Visser Leiden University
Pre-print Media Attached

Mon 31 May

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

09:00 - 10:00
Session 3: Software engineering lifecycle phases for AI systemsWAIN'21 at WAIN Room
Chair(s): Per Runeson Lund University
09:00
20m
Full-paper
Concepts in Testing of Autonomous Systems: Academic Literature and Industry Practice
WAIN'21
Qunying Song Lund University, Emelie Engstrom Lund University, Per Runeson Lund University
Pre-print Media Attached
09:20
20m
Full-paper
The Prevalence of Code Smells in Machine Learning projects
WAIN'21
Bart van Oort , Luís Cruz Deflt University of Technology, Maurício Aniche Delft University of Technology, Arie van Deursen Delft University of Technology, Netherlands
Pre-print Media Attached
09:40
10m
Short-paper
Systematic Mapping Study on the Machine Learning Lifecycle
WAIN'21
Yuanhao Xie , Luís Cruz Deflt University of Technology, Petra Heck Fontys ICT, Jan S. Rellermeyer TU Delft
Pre-print Media Attached
09:50
10m
Short-paper
Challenges and Governance Solutions for Data Science Services based on Open Data and APIs
WAIN'21
Pre-print Media Attached
10:00 - 10:15
Speed dating IIIWAIN'21 at WAIN Room
11:15 - 11:30
Virtual Coffee Break IIIWAIN'21 at WAIN Room
11:30 - 12:30
WAIN'21 Industry PanelWAIN'21 at WAIN Room
Chair(s): Helena Holmström Olsson Malmö University
11:30
60m
Talk
Industry panel
WAIN'21
P: Elena Fersman Ericsson & KTH Royal Institute of Technology, P: Roland Weiss ABB, P: Aleksander Fabijan Microsoft, A: Björn Brinne Peltarion, SE
Media Attached
12:30 - 13:30
14:30 - 14:45
Speed dating IVWAIN'21 at WAIN Room
14:45 - 15:00
Virtual Coffee Break VWAIN'21 at WAIN Room
15:00 - 16:00
Session 5: Designing AI systems IWAIN'21 at WAIN Room
Chair(s): Grace Lewis Carnegie Mellon Software Engineering Institute
15:00
20m
Full-paper
Requirement engineering challenges for AI-intense distributed systems
WAIN'21
Hans-Martin Heyn University of Gothenburg & Chalmers University of Technology, Eric Knauss Chalmers | University of Gothenburg, Amna Pir Muhammad
Pre-print Media Attached
15:20
20m
Full-paper
Adaptive Autonomy in Human-on-the-Loop Vision-Based Robotics Systems
WAIN'21
Sophia Abraham University of Notre Dame, Zachariah Carmichael University of Notre Dame, Rosaura VidalMata University of Notre Dame, Sreya Banerjee , Ankit Agrawal University of Notre Dame, Md Nafee Al Islam , Walter Scheirer , Jane Cleland-Huang University of Notre Dame
Pre-print Media Attached
15:40
20m
Full-paper
Software Architecture for ML-based Systems: What Exists and What Lies Ahead
WAIN'21
Henry Muccini University of L'Aquila, Italy, Karthik Vaidhyanathan University of L'Aquila
Pre-print Media Attached
16:00 - 16:15
Speed dating VWAIN'21 at WAIN Room
16:15 - 17:00
Session 6: Designing AI systems IIWAIN'21 at WAIN Room
Chair(s): Raghu Sangwan Pennsylvania State University
16:15
20m
Full-paper
Understanding and Modeling AI-Intensive System Development
WAIN'21
Luigi Lavazza Università degli Studi dell'Insubria, Sandro Morasca Università degli Studi dell'Insubria
Pre-print Media Attached
16:35
10m
Short-paper
Engineering an Intelligent Essay Scoring and Feedback System: An Experience Report
WAIN'21
Akriti Chadda , Raman Chandrasekar , Ian Gorton Northeastern University – Seattle, USA
Pre-print Media Attached
16:45
10m
Short-paper
Lessons Learned from Educating AI Engineers
WAIN'21
Petra Heck Fontys ICT
Pre-print Media Attached
17:00 - 18:00
WAIN'21 Research&Academic PanelWAIN'21 at WAIN Room
Chair(s): Jan Bosch Chalmers University of Technology
17:00
60m
Talk
Research&Academic Panel
WAIN'21
P: Jane Cleland-Huang University of Notre Dame, P: Ipek Ozkaya Carnegie Mellon Software Engineering Institute, P: Barbara Plank IT University of Copenhagen, DK, P: Brian Fitzgerald Lero - The Irish Software Research Centre and University of Limerick, P: Ian Gorton Northeastern University – Seattle, USA
Media Attached

Unscheduled Events

Not scheduled
Talk
Research&Academic Panel
WAIN'21

Accepted Papers

The accepted papers (full and short)

Title
Adaptive Autonomy in Human-on-the-Loop Vision-Based Robotics Systems
WAIN'21
Pre-print Media Attached
Challenges and Governance Solutions for Data Science Services based on Open Data and APIs
WAIN'21
Pre-print Media Attached
Characterizing and Detecting Mismatch in Machine-Learning-Enabled Systems
WAIN'21
Pre-print Media Attached
Concepts in Testing of Autonomous Systems: Academic Literature and Industry Practice
WAIN'21
Pre-print Media Attached
Corner Case Data Description and Detection
WAIN'21
Pre-print Media Attached
Data acquisition and the implication of machine learning in the development of a Clinical Decision Support system
WAIN'21
Media Attached
Data collection and Acceleration Infrastructure for FPGA-based Edge AI Applications
WAIN'21
Pre-print Media Attached
Engineering an Intelligent Essay Scoring and Feedback System: An Experience Report
WAIN'21
Pre-print Media Attached
Errol Koolmeister: Engineering AI at H&M group
WAIN'21
Media Attached
Integration of Convolutional Neural Networks in Mobile Applications
WAIN'21
Pre-print Media Attached
Lessons Learned from Educating AI Engineers
WAIN'21
Pre-print Media Attached
Linnaeus: A highly reusable and adaptable ML based log classification pipeline
WAIN'21
Media Attached
MLOps Challenges in Multi-Organization Setup: Experiences from Two Real-World Cases
WAIN'21
Pre-print Media Attached
Opening session
WAIN'21
Media Attached
Position paper: Why we need to align academic education and industry requirements with respect to Machine Learning
WAIN'21
Pre-print Media Attached
Practices for Engineering Trustworthy Machine Learning Applications
WAIN'21
Pre-print Media Attached
Product Engineering for Machine Learning: A Grey Literature Review
WAIN'21
Pre-print Media Attached
Requirement engineering challenges for AI-intense distributed systems
WAIN'21
Pre-print Media Attached
Robust Machine Learning in Critical Care - Software Engineering and Medical Perspectives
WAIN'21
Pre-print Media Attached
Software Architecture for ML-based Systems: What Exists and What Lies Ahead
WAIN'21
Pre-print Media Attached
Systematic Mapping Study on the Machine Learning Lifecycle
WAIN'21
Pre-print Media Attached
Technical Debt in Industrial AI Research Projects
WAIN'21
Pre-print Media Attached
The Prevalence of Code Smells in Machine Learning projects
WAIN'21
Pre-print Media Attached
Towards Productizing AI/ML Models: An Industry Perspective from Data Scientists.
WAIN'21
Pre-print Media Attached
Towards Risk Modeling for Collaborative AI
WAIN'21
Pre-print Media Attached
Understanding and Modeling AI-Intensive System Development
WAIN'21
Pre-print Media Attached
Verbatim Machine Learning Model Manifestation in Manageable Neighborhoods
WAIN'21
Pre-print Media Attached
Who Needs MLOps: What Data Scientists Seek to Accomplish and How Can MLOps Help?
WAIN'21
Pre-print Media Attached

ml-devops

In development and implementation of AI-based systems , the main challenge is not to develop the best models/algorithms, but to provide support for the entire lifecycle – from a business idea, through collection and management of data, software development managing both data and code, product deployment and operation, and to its evolution. There is a clear need for specific support of Software Engineering for AI.

The aim of the workshop is to bring together researchers and practitioners in software engineering, in data-science and AI, and to build up a community that will target the new challenges emerging in Software Engineering that AI/data-science engineers and software engineers are facing in development of AI-based systems. The workshop will be highly interactive: In addition to the invited keynotes and short paper presentations, there will be several discussion sessions. We plan to combine local and remote participation.

Call for submission

You are invited to submit

  • A research or experience full paper with 8 pages max. Papers describing the challenges, starting results, vision papers, or the experience papers from or in cooperation with the practitioners are encouraged.
  • A short research or experience paper with 4 pages max. The same topics as for long papers.
  • Position paper with expressed interest, 1 page.

The full and short paper submissions will undergo a review process with three independent reviews and a virtual PC decision meeting. The acceptance criteria include novelty, research and industrial relevance, soundness, experiences, and preliminary results. The accepted full and short papers will be published in IEEE Proceedings as a workshop proceedings at the ICSE conference. The position papers will be published on the workshop web page.

Topis of interests

The overall area is Software Engineering for AI, i.e. means to improve development of software AI-based systems and software-intensive systems, including topics relevant for the entire lifecycle. The suggested topics are (but not limited to):

  • System and software requirements and their relations AI/ML modelling;
  • Data management ensuring relevance and efficiency related to business goals;
  • System and software architecture of AI-based systems;
  • Integration of AI-development process and software development processes, including continuous and federated ML, continuous deployment, system and software evolution;
  • Ensuring and managing system and software nonfunctional properties and their relation to AI/ML properties, including run-time properties such as performance, safety, security, reliability, and life-cycle properties including reusability, maintainability and evolution;
  • Development teams, organizational and management issues for a successful development of AI-systems.

Submission form

Submissions must conform to the IEEE formatting instructions IEEE Conference Proceedings Formatting Guidelines. The official publication date of the workshop proceedings is the date the proceedings are made available by IEEE. This date may be up to two weeks prior to the first day of ICSE 2021. The official publication date affects the deadline for any patent filings related to published work.

Please note, the submissions should NOT be double blind, i.e. in the submission the authors should be specified.

The papers should be submitted to EasyChair web page. The submission deadline is firm.

Questions? Use the WAIN contact form.