Write a Blog >>
Tue 11 Oct 2022 11:10 - 11:30 at Ballroom C East - Technical Session 1 - AI for SE I Chair(s): Andrea Stocco

Aircraft industry is constantly striving for more efficient design optimization methods in terms of human efforts, computation time, and resources consumption. Hybrid surrogate optimization maintains high results quality while providing rapid design assessments when both the surrogate model and the switch mechanism for eventually transitioning to the HF model are calibrated properly. Feedforward neural networks (FNNs) can capture highly nonlinear input-output mappings, yielding efficient surrogates for aircraft performance factors. However, FNNs often fail to generalize over the out-of-distribution (OOD) samples, which hinders their adoption in critical aircraft design optimization. Through SmOOD, our smoothness-based out-of-distribution detection approach, we propose to codesign a model-dependent OOD indicator with the optimized FNN surrogate, to produce a trustworthy surrogate model with selective but credible predictions. Unlike conventional uncertainty-grounded methods, SmOOD exploits inherent smoothness properties of the HF simulations to effectively expose OODs through revealing their suspicious sensitivities, thereby avoiding over-confident uncertainty estimates on OOD samples. By using SmOOD, only high-risk OOD inputs are forwarded to the HF model for re-evaluation, leading to more accurate results at a low overhead cost. Three aircraft performance models are investigated. Results show that FNN-based surrogates outperform their Gaussian Process counterparts in terms of predictive performance. Moreover, SmOOD does cover averagely 85% of actual OODs on all the study cases. When SmOOD plus FNN surrogates are deployed in hybrid surrogate optimization settings, they result in a decrease error rate of 34.65% and a computational speed up rate of 58.36x, respectively.

Tue 11 Oct

Displayed time zone: Eastern Time (US & Canada) change

10:30 - 12:30
Technical Session 1 - AI for SE IResearch Papers / Industry Showcase at Ballroom C East
Chair(s): Andrea Stocco Università della Svizzera italiana (USI)
10:30
20m
Research paper
B-AIS: An Automated Process for Black-box Evaluation of AI-enabled Software Systems against Domain Semantics
Research Papers
Hamed Barzamini , Mona Rahimi Northern Illinois University
10:50
20m
Industry talk
Automatic Generation of Visualizations for Machine Learning Pipelines
Industry Showcase
Lei Liu Fujitsu Laboratories of America, Inc., Wei-Peng Chen Fujitsu Research of America, Inc., Mehdi Bahrami Fujitsu Laboratories of America, Inc., Mukul Prasad Amazon Web Services
11:10
20m
Research paper
SmOOD: Smoothness-based Out-of-Distribution Detection Approach for Surrogate Neural Networks in Aircraft DesignVirtual
Research Papers
Houssem Ben Braiek École Polytechnique de Montréal, Ali Tfaily Bombardier Aerospace, Foutse Khomh Polytechnique Montréal, Thomas Reid , Ciro Guida Bombardier Aerospace
Pre-print
11:30
20m
Research paper
Unveiling Hidden DNN Defects with Decision-Based Metamorphic TestingVirtual
Research Papers
Yuanyuan Yuan The Hong Kong University of Science and Technology, Qi Pang HKUST, Shuai Wang Hong Kong University of Science and Technology
11:50
20m
Research paper
Patching Weak Convolutional Neural Network Models through Modularization and CompositionVirtual
Research Papers
Binhang Qi Beihang University, Hailong Sun Beihang University, Xiang Gao Beihang University, China, Hongyu Zhang University of Newcastle
12:10
20m
Research paper
Safety and Performance, Why not Both? Bi-Objective Optimized Model Compression toward AI Software DeploymentVirtual
Research Papers
Jie Zhu Peking University, Leye Wang Peking University, Xiao Han Shanghai University of Finance and Economics
DOI Pre-print