TCSE logo 
 Sigsoft logo
Sustainability badge
Tue 29 Apr 2025 11:42 - 11:54 at 207 - Session 2 Chair(s): Foutse Khomh

Due to broad deployment of AI systems in risk-prone domains and AI regulations coming into effect, systematic risk- and quality assessments of AI systems have become increasingly important. Conducting such assessments involves identifying relevant quality criteria for a given AI system and selecting test methods, i.e., procedures for collecting and evaluating evidences and measurable quantities that fit the identified criteria. This selection process can be challenging due to the high complexity of the test method landscape and, in the context of independent audits, due to potential conflicts of interests between the involved stakeholders. To address this challenge, we expand existing frameworks on AI assessments with a systematic, taxonomy-based method of selecting test methods in independent audits suited for the given AI system and its application context. We evaluate our taxonomy on a subset of OECD metrics catalogue and demonstrate its applicability with two use cases.

Tue 29 Apr

Displayed time zone: Eastern Time (US & Canada) change

11:00 - 12:30
Session 2RAIE at 207
Chair(s): Foutse Khomh Polytechnique Montréal
11:00
15m
Talk
Using Drift Planning to Improve Safety of Visual Navigation in Unmanned Aerial Vehicles
RAIE
Jeffrey Hansen Carnegie Mellon Software Engineering Institute, Sebastian Echeverria Carnegie Mellon Software Engineering Institute, Lena Pons Carnegie Mellon Software Engineering Institute, Lihan Zhan Carnegie Mellon Software Engineering Institute, Gabriel A. Moreno Carnegie Mellon University Software Engineering Institute, P: Grace Lewis Carnegie Mellon Software Engineering Institute
11:15
15m
Talk
LLM-AQuA-DiVeR: LLM-Assisted Quality Assurance Through Dialogues on Verifiable Specification with Requirement Owners
RAIE
P: Shohei Mitani Georgetown University, Salonee Moona Triple Point Security, Shinichiro Matsuo Georgetown University, Eric Burger Virginia Tech
11:30
12m
Talk
Towards Ensuring Responsible AI for Medical Device Certification
RAIE
P: Giulio Mallardi University of Bari, Luigi Quaranta University of Bari, Italy, Fabio Calefato University of Bari, Filippo Lanubile University of Bari
11:42
12m
Talk
Navigating the landscape of AI test methods using taxonomy-based selection
RAIE
Maximilian Pintz Fraunhofer Institute for Intelligent Analysis and Information Systems, University of Bonn, Anna Schmitz Fraunhofer Institute for Intelligent Analysis and Information Systems, Rebekka Görge Fraunhofer Institute for Intelligent Analysis and Information Systems, P: Sebastian Schmidt Fraunhofer Institute for Intelligent Analysis and Information Systems, Daniel Becker , Maram Akila Fraunhofer Institute for Intelligent Analysis and Information Systems, Lamarr Institute, Michael Mock Fraunhofer Institute for Intelligent Analysis and Information Systems
11:54
12m
Talk
Responsible AI in the Software Industry: A Practitioner-Centered Perspective
RAIE
P: Matheus de Morais Leça University of Calgary, Mariana Pinheiro Bento University of Calgary, Ronnie de Souza Santos University of Calgary
Pre-print
12:06
12m
Talk
The Privacy Pillar - A Conceptual Framework for Foundation Model-based Systems
RAIE
Tingting Bi The University of Melbourne, Guangsheng Yu University of Technology Sydney, Qin Wang CSIRO Data61
:
:
:
: