OpenClassGen: A Large-Scale Corpus of Real-World Python Classes for LLM Research
Existing class-level code generation datasets are either synthetic (ClassEval: 100 classes) or insufficient in scale for modern training needs (RealClassEval: 400 classes), hindering robust evaluation and empirical analysis. We present OpenClassGen, a large-scale corpus of 324,843 Python classes extracted from 2,970 engineered open-source projects. Each entry pairs a human-written class with its corresponding skeleton, which comprises class and method signatures with associated docstrings, and is enriched with 27 static code metrics covering complexity, coupling, cohesion, and inheritance properties. Unlike prior benchmarks that require repository-level context resolution, OpenClassGen provides self-contained class skeletons that serve as complete generation specifications. We demonstrate the corpus’s utility by evaluating three LLMs (GPT-o4-mini, Claude-4-Sonnet, Qwen-3-Coder) on a curated, executable subset of 300 classes, enriched with test suites achieving 58% branch coverage. Results show strong semantic similarity (CodeBERTScore-F3: 0.89) but moderate functional correctness (pass rate: 0.33), with substantial variance across models. This variance, along with diverse class characteristics, confirms that OpenClassGen enables meaningful differentiation of LLM capabilities. The dataset supports diverse use cases, including fine-tuning, retrieval-augmented generation, difficulty modelling, and failure mode analysis. The complete dataset and curation scripts are publicly available at https://zenodo.org/records/18409150.
Wed 10 JunDisplayed time zone: London change
11:00 - 12:30 | LLMs for SE (Coding) 1Short Papers and Emerging Results / Posters and Vision / Industry Papers / AI Models / Data / Research Papers at JMS 743 Chair(s): Muhammad Waseem Faculty of Information Technology and Communication Sciences, Tampere University, 33014 Tampere, Finland | ||
11:00 15mTalk | OpenClassGen: A Large-Scale Corpus of Real-World Python Classes for LLM Research AI Models / Data Musfiqur Rahman Concordia University, Montreal, SayedHassan Khatoonabadi Concordia University, Emad Shihab Concordia University DOI Pre-print | ||
11:15 10mTalk | Leveraging LLMs for Multi-File DSL Code Generation: An Industrial Case Study Industry Papers Sivajeet Chand Technical University of Munich, Kevin Nguyen BMW Group, Peter Kuntz BMW Group, Alexander Pretschner TU Munich Pre-print | ||
11:25 15mTalk | Bridging the Programming Language Gap: Constructing a Multilingual Shared Semantic Space through AST Unification and Graph Matching Research Papers Junhao Chen Nanjing University of Aeronautics and Astronautics, Jingxuan Zhang Nanjing University of Aeronautics and Astronautics, Jian He Shanghai Aerospace Electronic Technology Institute, Yixuan Tang Nanjing University of Aeronautics and Astronautics, Weiqin Zou Nanjing University of Aeronautics and Astronautics Pre-print | ||
11:40 15mTalk | SelfHeal: Empirical Fix Pattern Analysis and Bug Repair in LLM Agents Research Papers Niful Islam Oakland University, Muhammad Anas Raza Oakland University, Mohammad Wardat Oakland University, USA Pre-print | ||
11:55 10mTalk | Quo Vadis, Code Review? Exploring the Future of Code Review Short Papers and Emerging Results Michael Dorner Technische Hochschule Nürnberg Georg Simon Ohm, Andreas Bauer Technische Hochschule Nürnberg Georg Simon Ohm, Darja Šmite Blekinge Institute of Technology, Lukas Thode Blekinge Institute of Technology, Daniel Mendez Blekinge Institute of Technology and fortiss, Ricardo Britto Ericsson / Blekinge Institute of Technology, Stephan Lukasczyk JetBrains Research, Ehsan Zabardast Nordea / Blekinge Institute of Technology, Michael Kormann SAP Pre-print | ||
12:05 10mTalk | To What Extent Does Agent-generated Code Require Maintenance? An Empirical Study Short Papers and Emerging Results Shota Sawada National Institute of Technology (KOSEN), Nara College, Tatsuya Shirai Nara Institute of Science and Technology, Yutaro Kashiwa Nara Institute of Science and Technology, Ken'Ichi Yamaguchi Nara National College of Technology, Hiroshi Iwata Nara National College of Technology, Hajimu Iida Nara Institute of Science and Technology | ||
12:15 10mTalk | GenAI in Software Engineering: The Role of Technology Acceptance Models Posters and Vision Oscar Johansson Blekinge Institute of Technology, Jürgen Börstler Blekinge Institute of Technology, Nauman bin Ali Blekinge Institute of Technology | ||