Write a Blog >>

Scene graph generation takes a camera image and derives a graph representation of key objects in the image and their relations. This core computer vision task is often used in autonomous driving, where traditional software and machine learning (ML) components are used in tandem. However, in such a safety-critical context, valid scene graphs can be further restricted by consistency constraints captured by domain or safety experts. However, existing ML approaches for scene graph generation focus exclusively on relational-level accuracy but provide little to no guarantee that consistency constraints are satisfied in the generated scene graphs. In this paper, we aim to complement existing ML-based approaches by a post-processing step using constraint optimization over probabilistic scene graphs that can (1) guarantee that no consistency constraints are violated and (2) improve the overall accuracy of the generated scene graphs by fixing constraint violations. We evaluate the effectiveness of our approach using well-known, and novel metrics in the context of two popular ML datasets augmented with consistency constraints and two ML-based scene graph generation approaches as baselines.

Thu 13 Oct

Displayed time zone: Eastern Time (US & Canada) change

13:30 - 15:30
Technical Session 28 - Safety-Critical and Self-Adaptive SystemsIndustry Showcase / Tool Demonstrations / Research Papers / Late Breaking Results / NIER Track at Room 128
Chair(s): Eunsuk Kang Carnegie Mellon University
SAFA: A Tool for Supporting Safety Analysis in Evolving Software Systems
Tool Demonstrations
Alberto D. Rodriguez University of Notre Dame, Timothy Newman University of Notre Dame, Katherine R. Dearstyne University of Notre Dame, Jane Cleland-Huang University of Notre Dame
Research paper
Generating Critical Test Scenarios for Autonomous Driving Systems via Influential Behavior PatternsVirtual
Research Papers
Haoxiang Tian Institute of Software, Chinese Academy of Sciences, Guoquan Wu Institute of Software at Chinese Academy of Sciences, China, Jiren Yan Institute of Software, Chinese Academy of Sciences, Yan Jiang Institute of Software, Chinese Academy of Sciences, Jun Wei Institute of Software at Chinese Academy of Sciences; University of Chinese Academy of Sciences, Wei Chen Institute of Software at Chinese Academy of Sciences, China, Shuo Li Institute of Software, Chinese Academy of Sciences, Dan Ye Institute of Software, Chinese Academy of Sciences
Research paper
Consistent Scene Graph Generation by Constraint OptimizationVirtual
Research Papers
Boqi Chen McGill University, Kristóf Marussy Budapest University of Technology and Economics, Sebastian Pilarski McGill University, Oszkár Semeráth Budapest University of Technology and Economics, Daniel Varro McGill University / Budapest University of Technology and Economics
Industry talk
A Drift Handling Approach for Self-Adaptive ML Software in Scalable Industrial ProcessesVirtual
Industry Showcase
Firas Bayram Department of Mathematics and Computer Science, Karlstad University, Sweden, Bestoun S. Ahmed Karlstad University Sweden, Erik Hallin Uddeholms AB, Sweden, Anton Engman Uddeholms AB, Sweden
SML4ADS: An Open DSML for Autonomous Driving Scenario Representation and GenerationVirtual
Late Breaking Results
Bo Li East China Normal University, Dehui Du East China Normal University, Sicong Chen East China Normal University, Minjun Wei East China Normal University, Chenghang Zheng East China Normal University, Xinyuan Zhang East China Normal University
Vision and Emerging Results
XSA: eXplainable Self-AdaptationVirtual
NIER Track
Matteo Camilli Free University of Bozen-Bolzano, Raffaela Mirandola Politecnico di Milano, Patrizia Scandurra University of Bergamo, Italy
File Attached
Industry talk
Design-Space Exploration for Decision-Support Software
Industry Showcase
Ate Penders Thales Research & Technology, Ana Lucia Varbanescu University of Twente, Gregor Pavlin Thales Research & Technology, Henk Sips Delft University of Technology