SANER 2025
Tue 4 - Fri 7 March 2025 Montréal, Québec, Canada
Thu 6 Mar 2025 11:45 - 12:00 at M-1410 - Program Analysis Chair(s): Rrezarta Krasniqi

With the increasing popularity of LLM-based code completers, like GitHub Copilot, the interest in automatically detecting AI-generated code is also increasing—in particular in contexts where the use of LLMs to program is forbidden by policy due to security, intellectual property, or ethical concerns. We introduce a novel technique for AI code stylometry, i.e., the ability to distinguish code generated by LLMs from code written by humans, based on a transformer-based encoder classifier. Differently from previous work, our classifier is capable of detecting AI-written code across 10 different programming languages with a single machine learning model, maintaining high average accuracy across all languages (84.1% ± 3.8%). Together with the classifier we also release H-AIRosettaMP, a novel open dataset for AI code stylometry tasks, consisting of 121 247 code snippets in 10 popular programming languages, labeled as either human-written or AI-generated. The experimental pipeline (dataset, training code, resulting models) is the first fully reproducible one for the AI code stylometry task. Most notably our experiments rely only on open LLMs, rather than on proprietary/closed ones like ChatGPT.

Thu 6 Mar

Displayed time zone: Eastern Time (US & Canada) change

11:00 - 12:30
Program AnalysisResearch Papers at M-1410
Chair(s): Rrezarta Krasniqi University of North Carolina at Charlotte
11:00
15m
Talk
Adapting Knowledge Prompt Tuning for Enhanced Automated Program Repair
Research Papers
Xuemeng Cai Singapore Management University, Lingxiao Jiang Singapore Management University
Pre-print
11:15
15m
Talk
A Metric for Measuring the Impact of Rare Paths on Program Coverage
Research Papers
Leo St. Amour Virginia Tech, Eli Tilevich Virginia Tech, Muhammad Ali Gulzar Virginia Tech
11:30
15m
Talk
A Progressive Transformer for Unifying Binary Code Embedding and Knowledge Transfer
Research Papers
Hanxiao Lu Columbia University, Hongyu Cai Purdue University, Yiming Liang Purdue University, Antonio Bianchi Purdue University, Z. Berkay Celik Purdue University
11:45
15m
Talk
Is This You, LLM? Recognizing AI-written Programs with Multilingual Code Stylometry
Research Papers
Andrea Gurioli DISI - University of Bologna, Maurizio Gabbrielli DISI - University of Bologna, Stefano Zacchiroli Télécom Paris, Polytechnic Institute of Paris
Pre-print
12:00
15m
Talk
SpeedGen: Enhancing Code Efficiency through Large Language Model-Based Performance Optimization
Research Papers
Nils Purschke Technical University of Munich, Sven Kirchner Technical University of Munich, Alois Knoll Technical University of Munich
12:15
15m
Talk
StriCT-BJ: A String Constraint Benchmark from Real Java Programs
Research Papers
Chi Zhang Institute of Software at Chinese Academy of Sciences; University of Chinese Academy of Sciences, Jian Zhang Institute of Software at Chinese Academy of Sciences; University of Chinese Academy of Sciences