A Case Study on AI Engineering Practices: Developing an Autonomous Stock Trading System
Today, many systems use artificial intelligence (AI) to solve complex problems. While this often increases system effectiveness, developing a production-ready AI-based system is a difficult task. Thus, solid AI engineering practices are required to ensure the quality of the resulting system and to improve the development process. While several practices have already been proposed for the development of AI-based systems, detailed practical experiences of applying these practices are rare.
In this paper, we aim to address this gap by collecting such experiences during a case study, namely the development of an autonomous stock trading system that uses machine learning functionality to invest in stocks. We selected 10 AI engineering practices from the literature and systematically applied them during development, with the goal to collect evidence about their applicability and effectiveness. Using structured field notes, we documented our experiences. Furthermore, we also used field notes to document challenges that occurred during the development, and the solutions we applied to overcome them. Afterwards, we analyzed the collected field notes, and evaluated how each practice improved the development. Lastly, we compared our evidence with existing literature.
Most applied practices improved our system, albeit to varying extent, and we were able to overcome all major challenges. The qualitative results provide detailed accounts about 10 AI engineering practices, as well as challenges and solutions associated with such a project. Our experiences therefore enrich the emerging body of evidence in this field, which may be especially helpful for practitioner teams new to AI engineering.
Tue 16 MayDisplayed time zone: Hobart change
20:15 - 21:45
|A Case Study on AI Engineering Practices: Developing an Autonomous Stock Trading System
|Engineering Challenges for AI-Supported Computer Vision in Small Uncrewed Aerial Systems
|A Meta-Summary of Challenges in Building Products with ML Components -- Collecting Experiences from 4758+ PractitionersDistinguished paper Award Candidate
Nadia Nahar Carnegie Mellon University, Haoran Zhang Carnegie Mellon University, USA, Grace Lewis Carnegie Mellon Software Engineering Institute, Shurui Zhou Carnegie Mellon University, USA / University of Toronto, CA, Christian Kästner Carnegie Mellon UniversityPre-print File Attached
|Design Patterns for AI-based Systems: A Multivocal Literature Review and Pattern Repository
Lukas Heiland University of Stuttgart, Germany, Marius Hauser University of Stuttgart, Germany, Justus Bogner University of StuttgartPre-print