The rise of ChatGPT and GitHub Copilot has sparked a surge in developers leveraging large language models (LLMs) for code generation, aiming to automate software development processes. However, these tools can generate substandard and vulnerable code. Notably, a significant portion of developers in the US embrace LLMs due to productivity boost. However, research indicates that LLM-generated code may compromise security, with users often overestimating its reliability. To address these challenges, this proposal aims to enhance the quality and security of generated code in outputs. The proposal includes an empirical study of code generation models’ training sets and benchmarks for code and security smells. It also consists of a framework, SALLM, to automatically benchmark code generation models from the security perspective. This proposal is a work in progress in creating quality datasets to reinforce the code generation model and generate standard and secure code. By establishing trust in LLM-based tools and generating secure and standard code, developers can confidently integrate them into their workflows and rely on them.

I am a Ph.D. candidate working as a Graduate Research Assistant at the University of Notre Dame, IN, USA. I mainly focus on Code Generation Models and their applications in software testing and security.