This presentation describes the challenges related to the use of AI in safety-critical automotive applications (for example, automated driving). An overview of the upcoming standard ISO PAS 8800 Road Vehicles - Safety and AI will be provided, as well as the wider context of safety standards in this area (ISO 26262, ISO 21448, ISO TS 5083). The presentation will then motivate the structure and key concepts of ISO PAS 8800 and present how existing approaches to safety of automotive E/E systems are extended for AI and ML-based functions. This includes the consideration of AI-based functions within the wider system context, the derivation of AI safety requirements, data considerations, safety analysis and test.

The presentation will then focus on open research challenges. In particular, the need to address uncertainty in the assurance arguments for such systems and how this uncertainty can be evaluated and reduced. This includes determining the relevance of popular machine learning metrics and understanding their actual contribution to a rigorous safety assurance argument. The presentation is intended to provoke a dialogue between various application domains of safety-critical AI to compare and contrast the associated technical requirements, associated safety risks and assurance approaches.