Evaluating AI-based Code Segmentation for ABAP Programs in an Industrial Use Case
Many maintenance and evolution tasks in software engineering depend on the availability of logical code segments, especially AI- and data-driven approaches rely on segmented source code. In order to obtain logical segments a manual step is typically necessary. However, manual segmentation requires a basic understanding of the source code and delays the application of code analysis and refactoring tools. Automatic code segmentation provides an efficient way of extracting code snippets for further analysis to provide developers with actionable insights on software products and processes. Rule-based approaches rely on syntactic boundaries and lack the applicability of segmentation on multiple languages. In this article, we present our approach to learning logical code snippets using a BiLSTM neural network model. Driven by the requirements of an industrial use case, we train two models, one on Go, Java, JavaScript, Python, PHP, and Ruby, the other model is additionally trained on ABAP snippets. To evaluate the performance of the models, we use real-world samples used in the SAP applications of our industry partner. We also compare the predictions of the model, with a performance >98%, to the results of human experts for segmenting ABAP code to evaluate whether AI-based code segmentation is perceived as effective by practitioners in our industrial use case. The study shows that only 42-51% of the predicted ABAP code snippets match the manual segmentation of the experts.
Wed 4 DecDisplayed time zone: Athens change
11:00 - 12:30 | PROFES Session 7: AI for Software Engineering in Practice (II)Research Papers / Industry Papers at UT Library - Room 2 (Seminar Room Tõstamaa) Chair(s): Stefan Sauer Paderborn University | ||
11:00 18mResearch paper | Enhancing Productivity with AI During the Development of an ISMS: Case Kempower Research Papers Atro Niemeläinen Kempower, Muhammad Waseem University of Jyväskylä, Jyväskylä, Finland, Tommi Mikkonen University of Jyvaskyla | ||
11:18 18mResearch paper | Experience with Large Language Model Applications for Information Retrieval from Enterprise Proprietary Data Research Papers Liang Yu Blekinge Institute of Technology, Emil Alégroth Blekinge Institute of Technology, Panagiota Chatzipetrou , Tony Gorschek Blekinge Institute of Technology / DocEngineering | ||
11:36 18mIndustry talk | Evaluating AI-based Code Segmentation for ABAP Programs in an Industrial Use Case Industry Papers Richard Mayer Software Competence Center Hagenberg GmbH, Michael Moser Software Competence Center Hagenberg GmbH, Niklas Greif Sysparency GmbH, Florian Schnitzhofer Sysparency GmbH, Verena Geist Software Competence Center Hagenberg GmbH, Martin Pinzger Universität Klagenfurt | ||
11:54 36mTalk | Session 7 Discussion Research Papers |