Write a Blog >>

Abstract: Testing cyber-physical systems presents a unique set of testing challenges: heterogeneity, timing, and, especially, observability. In particular, some of the mechanisms that are designed to make embedded software robust are the same mechanisms that present challenges for automated testing techniques: e.g., rate limiting, fault masking, and debounce logic, which can lead to long lags between problematic inputs and their manifestation in system outputs. In addition, much of the control behavior of CPS is mathematically intensive more than “branchy”. For such systems, traditional coverage metrics, which focus on reaching program statements or mutants (as in weak mutation), rather than observing their affect on system outputs, are likely to lead to poor fault-finding in practice. I will describe some recent work in the area of test generation and code metrics that begins to address these issues.

I then argue that this mismatch will become significantly worse for when examining CPS systems that incorporate machine learning: the complexity of the non-linear mathematics used defeats all known symbolic techniques and the lack of a traditional branching structure renders traditional code coverage metrics fairly meaningless when using search-based testing. In fact, a recent paper abstract claims: “We propose a novel control flow obfuscation design based on the incomprehensibility of artificial neural networks to fight against reverse engineering tools including concolic testing.” Traditional testing of such systems has focused on checking their behavior against a large corpus of examples; unfortunately, it is known that such systems are often not robust against malicious inputs even after extensive training and testing. I hope to challenge participants to focus research into such mathematically intensive systems and to begin to create rigorous metrics that are applicable to such systems.

Biography: Dr. Michael Whalen is the Director of the University of Minnesota Software Engineering Center and the Director of Graduate Studies for the Masters of Science in Software Engineering (MSSE) program. He has more than 20 years of experience in software development and analysis, including 15 years of experience in Model-Based Development & safety-critical systems. Dr. Whalen has developed simulation, translation, testing, and formal analysis tools for Model-Based Development languages including Simulink, Stateflow, Lustre, and RSML-e. He has led successful large-scale formal verification projects on industrial avionics systems, including displays (Rockwell-Collins ADGS-2100 Window Manager, deployed on the Boeing 787), redundancy management and control allocation (AFRL CerTA FCS program) and autoland (AFRL CerTA CPD program). Dr. Whalen was the lead developer of the Rockwell-Collins Gryphon tool suite, which was used for compilation, test-case generation, and formal analysis of Simulink/Stateflow models. He is currently working on the AGREE tool suite for architectural analysis and testing in AADL, which was recently used to verify correct system behavior in the Boeing Little Bird autonomous helicopter as part of the DARPA HACMS project.

ISSTA registration is open! See the registration and accommodation info. Hope to see you in Santa Barbara in July!

Check out our week overview!

SPIN, ISSTA, and RERS registration is now open! Visit the ISSTA Registration site to register and reserve your accommodations. On-campus accommodations, both shared and private, are available, but limited in number. So hurry to book them! Nearby off-campus choices are also available.

SPIN will be held at the University of California, Santa Barbara. For information on the venue and travel to Santa Barbara, visit this ISSTA page.

This year we accepted 16 full and 5 short papers out of 46 submissions. You can find the list of accepted papers here.

We have extended the paper submission deadline by one week to Friday, February 17. We have also pushed back the author notification date to April 20, 2017. There is no change to the Camera Ready version deadline.

SPIN organizers will accommodate, via alternative means of presentation, the authors of accepted papers if they are affected by the recent travel restrictions to the USA.

Our organizational sponsor, Association of Computing Machinery (ACM), has provided a statement on this issue. You may view it here.

We are pleased to announce Domagoj Babic (Google), Byron Cook (Amazon Web Services), Gerard Holzmann (Nimble Research) as our keynote speakers.

Details to follow…

We are excited to announce the first keynote speaker for SPIN 2017: Dr. Domagoj Babic from Google, Inc.

Domagoj Babic

Domagoj Babic is a computer scientist at Google, Inc. His work focuses on research and development of automated software analysis systems for various security-related applications. Primarily, he wants his work to have a positive impact on people’s lives. He enjoys building strong teams and working with them on solving large-scale real-world important problems, while learning and having fun on the way. He’s particularly excited about big technical challenges and enjoy creating the vision, strategy, and technology for taming those challenges. Over his career, Domagoj has published in the areas of verification, testing, security of complex software systems, automated reasoning, grammar inference, and applied formal methods. Before joining Google, Domagoj was a research scientist at UC Berkeley and elsewhere in industry. He received his Dipl.Ing. in Electrical Engineering and M.Sc. in Computer Science from the Zagreb University (Faculty of Electrical Engineering and Computing) in 2001 and 2003. He received his Ph.D. in Computer Science in 2008 from the University of British Columbia. He was a recipient of the NSERC PDF Research Fellowship (2010-2012), Microsoft Graduate Research Fellowship (2005-2007), and several awards at international programming competitions (1st place at the 2007 Satisfiability Modulo Theories competition in the bit-vector arithmetic category and 3rd place at the 2005 Satisfiability Testing competition in the satisfiable-crafted instances category).

Mike Whalen will give the keynote address at TECPSTue 20 Jun 2017
ISSTA registration is open!Tue 9 May 2017
ISSTA week overview is onlineFri 5 May 2017
SPIN registration is now open!Thu 4 May 2017
SPIN accepted papers are available!Sat 22 Apr 2017
SPIN submission deadline extended to Feb 17!Tue 7 Feb 2017
Statement on US Travel RestrictionsThu 2 Feb 2017
SPIN keynote speakers announced!Fri 20 Jan 2017
Domagoj Babic from Google will give a keynote address at SPIN 2017Thu 12 Jan 2017