Towards Concrete and Connected AI Risk Assessment (C2AIRA): A Systematic Mapping Study
Sat 20 May 2023 14:02 - 14:10 at Meeting Room 105 - Realizing the Promise of AI: Challenges and Visions Chair(s): Ipek Ozkaya
The rapid development of artificial intelligence (AI) has led to increasing concerns about the capability of AI systems to make decisions and behave responsibly. Responsible AI (RAI) refers to the development and use of AI systems that benefit humans, society, and the environment while minimising the risk of negative consequences. To ensure responsible AI, the risks associated with AI systems’ development and use must be identified, assessed and mitigated. Various AI risk assessment frameworks have been released recently by governments, organisations, and companies. However, it can be challenging for AI stakeholders to have a clear picture of the available frameworks and determine the most suitable ones for a specific context. Additionally, there is a need to identify areas that require further research or development of new frameworks, as well as updating and maintaining existing ones. To fill the gap, we present a mapping study of 16 existing AI risk assessment frameworks from the industry, governments, and non-government organizations (NGOs). We identify key characteristics of each framework and analyse them in terms of RAI principles, stakeholders, system lifecycle stages, geographical locations, targeted domains, and assessment methods. Our study provides a comprehensive analysis of the current state of the frameworks and highlights areas of convergence and divergence among them. We also identify the deficiencies in existing frameworks and outlines the essential characteristics of a concrete and connected framework AI risk assessment (C2AIRA) framework. Our findings and insights can help relevant stakeholders choose suitable AI risk assessment frameworks and guide the design of future frameworks towards concreteness and connectedness.
Tue 16 MayDisplayed time zone: Hobart change
Sat 20 MayDisplayed time zone: Hobart change
13:30 - 15:00 | Realizing the Promise of AI: Challenges and Visions Papers at Meeting Room 105 Chair(s): Ipek Ozkaya Carnegie Mellon University | ||
13:30 8mLong-paper | A Meta-Summary of Challenges in Building Products with ML Components -- Collecting Experiences from 4758+ PractitionersDistinguished paper Award Candidate Papers Nadia Nahar Carnegie Mellon University, Haoran Zhang Carnegie Mellon University, USA, Grace Lewis Carnegie Mellon Software Engineering Institute, Shurui Zhou University of Toronto, Canada, Christian Kästner Carnegie Mellon University Pre-print File Attached | ||
13:38 8mShort-paper | Dataflow graphs as complete causal graphs Papers Andrei Paleyes Department of Computer Science and Technology, Univesity of Cambridge, Siyuan Guo Max Planck Institute for Intelligent Systems, Bernhard Schölkopf MPI Tuebingen, Neil D. Lawrence Department of Computer Science and Technology, Univesity of Cambridge Pre-print | ||
13:46 8mShort-paper | Prevalence of Code Smells in Reinforcement Learning Projects Papers Nicolás Cardozo Universidad de los Andes, Ivana Dusparic Trinity College Dublin, Ireland, Christian Cabrera Department of Computer Science and Technology, Univesity of Cambridge Pre-print Media Attached | ||
13:54 8mShort-paper | Towards Code Generation from BDD Test Case Specifications: A vision Papers Leon Chemnitz TU Darmstadt, David Reichenbach TU Darmstadt, Germany, Hani Aldebes TU Darmstadt, Mariam Naveed TU Darmstadt, Krishna Narasimhan TU Darmstadt, Mira Mezini TU Darmstadt Pre-print | ||
14:02 8mLong-paper | Towards Concrete and Connected AI Risk Assessment (C2AIRA): A Systematic Mapping Study Papers Boming Xia CSIRO's Data61 & University of New South Wales, Qinghua Lu CSIRO’s Data61, Harsha Perera CSIRO's Data61 & University of New South Wales, Liming Zhu The University of New South Wales, Zhenchang Xing , Yue Liu CSIRO's Data61 & University of New South Wales, Jon Whittle CSIRO's Data61 and Monash University Pre-print | ||
14:10 50mPanel | Panel Discussion - Onsite Papers |