ICSE 2026
Sun 12 - Sat 18 April 2026 Rio de Janeiro, Brazil

Our work examines how Large Language Models can be systematically and reliably used to detect code smells, moving beyond zero-shot assumptions toward a reproducible, evidence-driven approach. Using GPT-4o and a curated oracle derived from the CrowdSmelling dataset, we study three representative smells in Java, Long Method, God Class, and Feature Envy, and evaluate how performance changes as we progressively refine the prompting strategy and the contextual information provided to the model. The study evolved through four stages. We began with a simple baseline in which the prompt only mentioned the smell name, testing whether the model could identify smells from prior knowledge alone. We then introduced specialized prompts containing structured definitions of each smell, showing that clearer task framing and explicit criteria significantly improve classification. In the third stage, we augmented prompts with quantitative code metrics alongside the source code, giving the model objective signals related to size, complexity, coupling, cohesion, and behavioral patterns. Finally, we applied hyperparameter tuning to achieve more stable, consistent outputs and to assess whether increased determinism could yield additional gains. The progressive refinement of our setup significantly improved detection performance, with the best configuration reaching AUC 0.82, F1 Score 0.78, and Accuracy 0.75, showing that prompt and context design can be as influential as the model itself. Results also varied by smell type: God Class, a mostly structural smell, benefited strongly from metric-enriched prompts and was perfectly classified, whereas Feature Envy remained challenging due to subtle cross-class context, often leading to false positives. This suggests the need for richer representations and more targeted metric selection for harder smells. We are extending the study with the DACOS benchmark, which includes 10,267 manual annotations for 5,192 code snippets and focuses on three additional code smells: Multifaceted Abstraction, Complex Method, and Long Parameter List, while emphasizing subjective borderline cases to validate further and refine our approach. We will also run the same protocol with a newer model, such as GPT-5, and compare it to GPT-4o under identical settings to separate model gains from prompt and context effects, and to assess whether newer LLMs reduce false positives and improve the detection of context-dependent smells.