CIgrate: Automating CI Service Migration with Large Language Models
Continuous Integration (CI) configurations often need to be migrated between services (e.g., Travis CI to GitHub Actions) as projects evolve, due to changes in service capabilities, usage limits, or service deprecation. Previous studies reported that migration across CI services is a recurring need in open-source development. However, manual migration can be time-consuming and error-prone. The state-of-the-art approach, CIMig, addresses this challenge by analyzing past migration examples to create service-specific rules and produce equivalent configurations across CI services. However, its relatively low accuracy raises concerns about the overall feasibility of automated CI migration using rule-based techniques alone. Meanwhile, Large Language Models (LLMs) have demonstrated strong capabilities in code generation and transformation tasks, suggesting potential to improve the automation, usability, and generalizability of CI configuration migration. This registered report presents a study in which we aim to assess whether CI migration can be improved using LLMs. To this end, we propose CIgrate, an LLM-based framework for automatically migrating CI configurations. We plan to evaluate the performance of CIgrate compared to CIMig as a baseline, in different setups (a) zero-shot/few-shot prompting of LLMs for configuration migration and (b) fine-tuning an LLM on a dataset of already established CI service migrations. We will also seek developer feedback on the quality and usability of the generated configurations. We formulate research questions focusing on the accuracy of LLM-generated migrations versus ground truth and the output of CIMig. The expected contributions include the first LLM-powered approach for CI service migration, a comparative evaluation of its effectiveness compared to rule-based approaches, and insight into leveraging LLMs to support software configuration evolution.
Fri 12 SepDisplayed time zone: Auckland, Wellington change
10:30 - 12:00 | Session 13 - Reuse 1NIER Track / Research Papers Track / Industry Track / Registered Reports at Case Room 3 260-055 Chair(s): Banani Roy University of Saskatchewan | ||
10:30 15m | From Release to Adoption: Challenges in Reusing Pre-trained AI Models for Downstream Developers Research Papers Track Peerachai Banyongrakkul The University of Melbourne, Mansooreh Zahedi The Univeristy of Melbourne, Patanamon Thongtanunam University of Melbourne, Christoph Treude Singapore Management University, Haoyu Gao The University of Melbourne Pre-print | ||
10:45 15m | Are Classical Clone Detectors Good Enough For the AI Era? Research Papers Track Ajmain Inqiad Alam University of Saskatchewan, Palash Ranjan Roy University of Saskatchewan, Farouq Al-Omari Thompson Rivers University, Chanchal K. Roy University of Saskatchewan, Banani Roy University of Saskatchewan, Kevin Schneider University of Saskatchewan | ||
11:00 10m | Can LLMs Write CI? A Study on Automatic Generation of GitHub Actions Configurations NIER Track Taher A. Ghaleb Trent University, Dulina Rathnayake Department of Computer Science, Trent University, Peterborough, Canada Pre-print | ||
11:10 10m | A Preliminary Study on Large Language Models Self-Negotiation in Software Engineering NIER Track Chunrun Tao Kyushu University, Honglin Shu Kyushu University, Masanari Kondo Kyushu University, Yasutaka Kamei Kyushu University | ||
11:20 10m | CIgrate: Automating CI Service Migration with Large Language Models Registered Reports Md Nazmul Hossain Department of Computer Science, Trent University, Peterborough, Canada, Taher A. Ghaleb Trent University Pre-print | ||
11:30 15m | A Deep Dive into Retrieval-Augmented Generation for Code Completion: Experience on WeChat Industry Track Zezhou Yang Tencent Inc., Ting Peng Tencent Inc., Cuiyun Gao Harbin Institute of Technology, Shenzhen, Chaozheng Wang The Chinese University of Hong Kong, Hailiang Huang Tencent Inc., Yuetang Deng Tencent | ||
11:45 10m | Inferring Attributed Grammars from Parser Implementations NIER Track Andreas Pointner University of Applied Sciences Upper Austria, Hagenberg, Austria, Josef Pichler University of Applied Sciences Upper Austria, Herbert Prähofer Johannes Kepler University Linz Pre-print | ||