A Preliminary Study on the Vocabulary of Flaky Tests in Swift
Flaky tests yield inconsistent results without any code changes, damaging the confidence in test suites. This work investigates the feasibility of detecting flaky tests in the Swift programming language through static analysis techniques combined with machine learning. From 14 open-source projects, we collected flaky tests through systematic re-execution and commit analysis, extracted tokens, and trained five classifiers (Random Forest, Decision Tree, Naive Bayes, SVM, and KNN) with TF-IDF vectors. The Random Forest classifier achieved the best results, followed by SVM, demonstrating that it is possible to predict instability with high accuracy based solely on the code. The results confirm the effectiveness of static detection and vocabulary as predictors of flakiness within the Swift ecosystem, aiding developers in proactively identifying these issues.
Mon 13 AprDisplayed time zone: Brasilia, Distrito Federal, Brazil change
16:00 - 17:30 | |||
16:00 20mTalk | A Preliminary Study on the Vocabulary of Flaky Tests in Swift FTW | ||
16:20 20mTalk | Flaky Tests in a Large Industrial Database Management System: An Empirical Study of Fixed Issue Reports for SAP HANA FTW Pre-print | ||
16:40 20mTalk | Preliminary Results on Evaluating Large Language Models for Labeling Root Cause Categories of Fixed Flaky Tests FTW Yang Chen University of Illinois at Urbana-Champaign, Kaiyao Ke University of California Berkeley, Darko Marinov University of Illinois at Urbana-Champaign | ||
17:00 30mPanel | Panel: Future of Flaky Test Research in the Era of Generative AI FTW | ||