ICSA 2026
Mon 22 - Fri 26 June 2026

This program is tentative and subject to change.

Thu 25 Jun 2026 14:30 - 14:45 at Theatre 2 - Benchmarks & Smells

Benchmarks for large language models (LLMs) have progressed from snippet-level function generation to repository-level issue resolution, yet they overwhelmingly target implementation correctness. Software architecture tasks remain under-specified and difficult to compare across models, despite their central role in maintaining and evolving complex systems. We present ArchBench, the first unified platform for benchmarking LLM capabilities on software architecture tasks. ArchBench provides a command-line tool with a standardized pipeline for dataset download, inference with trajectory logging, and automated evaluation, alongside a public web interface with an interactive leaderboard. The platform is built around a plugin architecture where each task is a self-contained module, making it straightforward for the community to contribute new architectural tasks and evaluation results. We use the term LLMs broadly to encompass generative AI (GenAI) solutions for software engineering, including both standalone models and LLM-based coding agents equipped with tools. Both the CLI tool and the web platform are openly available to support reproducible research and community-driven growth of architectural benchmarking.

This program is tentative and subject to change.

Thu 25 Jun

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

14:30 - 15:00
14:30
15m
Paper
ArchBench: Benchmarking Generative-AI for Software Architecture Tasks
Software Architecture Showcase
Bassam Adnan , Aviral Gupta SERC, IIIT Hyderabad, India, Sreemaee Akshathala IIIT Hyderabad, Karthik Vaidhyanathan IIIT Hyderabad
14:45
15m
Paper
Rethinking MVC Violations as Security-Aware Architectural Smells in Android Applications
New and Emerging Ideas
Mohamed Sylla , Karim Elish Florida Polytechnic University