Hybrid Deep Neural Networks to Infer State Models of Black-Box Systems
Specification mining, in general, and inferring behavior model of a running system, in particular, are quite useful for several automated software engineering tasks, such as program comprehension, anomaly detection, and testing. Most existing dynamic model inference techniques are white-box, i.e., they require source code to be instrumented to get run-time traces. However, in many systems, access to source code is not possible for parts of the program that use third-party binaries and off-the-shelf-components. One useful scenario for automated black-box behaviour inference is in software control units (such as autopilots), where the software system’s reactions over time changes based on the inputs. Run-time state models of such systems are very powerful means for anomaly detection and debugging. Unfortunately, most black-box techniques that detect state changes over time are either uni-variate (which is limiting the application in real-world systems) or are weak with respect to learning from past behaviour. Therefore, in this paper, we propose a hybrid deep neural network that accepts as input a set of time series, one per input signal of the system, and applies a set of convolution and recurrent layers to both learn the non-linear correlations between signals and the patterns over time. We have applied our approach to a real UAV auto-pilot solution from our industry partner with half a million lines of C code. We ran 888 random recent test cases of the system and inferred states over time. We compared our results with several traditional time series change point detection techniques and showed that our approach can improve their performance 88% to 102%, in terms of finding state change points, measured by F1 score. We also showed that our state classification algorithm provides on average 90.45% F1 score, which improves traditional classification algorithms 7% to 17%.
Tue 22 SepDisplayed time zone: (UTC) Coordinated Universal Time change
17:10 - 18:10 | AI for Software Engineering (1)NIER track / Research Papers at Koala Chair(s): Tingting Yu University of Kentucky | ||
17:10 20mTalk | DeepTC-Enhancer: Improving the Readability of Automatically Generated Tests Research Papers Devjeet Roy Washington State University, Ziyi Zhang Washington State University, Maggie Ma Washington State University, Venera Arnaoudova Washington State University, Annibale Panichella Delft University of Technology, Sebastiano Panichella Zurich University of Applied Sciences, Danielle Gonzalez Rochester Institute of Technology, USA, Mehdi Mirakhorli Rochester Institute of Technology | ||
17:30 20mTalk | Hybrid Deep Neural Networks to Infer State Models of Black-Box Systems Research Papers Pre-print | ||
17:50 10mTalk | On Benign Features in Malware Detection NIER track Michael Cao The University of British Columbia, Sahar Badihi University of British Columbia, Canada, Khaled Ahmed The University of British Columbia, Peiyu Xiong The University of British Columbia, Julia Rubin University of British Columbia, Canada |