APSEC 2022
Tue 6 - Fri 9 December 2022

Three keynote speakers have been appointed as follows.

December 7th

Dr. Hiroshi Maruyama (Kao Corporation, Executive Fellow. The University of Tokyo, Project Professor. Preferred Networks, Inc., Fellow)

December 8th

Dr. Grace A. Lewis (Carnegie Mellon University Software Engineering Institute (SEI), Lead for the SEI Tactical and AI-enabled Systems initiative. IEEE Computer Society, Vice President)

December 9th

Prof. Shing-Chi Cheung (Hong Kong University of Science and Technology, Professor)

Dates
Tracks
Plenary
You're viewing the program in a time zone which is different from your device's time zone change time zone

Tue 6 Dec

Displayed time zone: Osaka, Sapporo, Tokyo change

15:00 - 16:00
TutorialTutorial at Hall
15:00
60m
Tutorial
Process Mining for Software Engineers
Tutorial
Wil van der Aalst RWTH Aachen University, Germany

Wed 7 Dec

Displayed time zone: Osaka, Sapporo, Tokyo change

13:00 - 14:00
Empirical Studies 1SEIP - Software Engineering in Practice / Technical Track at Room1
Chair(s): Masateru Tsunoda Kindai University
13:00
20m
Paper
An Empirical Study on Multi-Source Cross-Project Defect Prediction Models
Technical Track
Xuanying Liu Beijing Jiaotong University, Zonghao Li Beijing Jiaotong University, Jiaqi Zou Beijing Jiaotong University, Haonan Tong Beijing Jiaotong University
13:20
15m
Paper
Refactoring Community Smells: An Empirical Study on the Software Practitioners of Bangladesh
SEIP - Software Engineering in Practice
Noshin Tahsin Institute of Information Technology, University of Dhaka, Kazi Sakib Institute of Information Technology, University of Dhaka
13:35
20m
Paper
How Libraries Evolve: A Survey of Two Industrial Companies and an Open-Source Community
Technical Track
Oleksandr Zaitsev Arolla, Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189 CRIStAL, F-59000 Lille, France, Stéphane Ducasse Inria; University of Lille; CNRS; Centrale Lille; CRIStAL, Nicolas Anquetil University of Lille, Lille, France, Arnaud Thiefaine Arolla
13:00 - 14:00
Machine Learning 1Technical Track at Room2
Chair(s): Syful Islam Nara Institute of Science and Technology
13:00
20m
Paper
Catch Me If You Can: Blackbox Adversarial Attacks on Automatic Speech Recognition using Frequency Masking
Technical Track
Xiaoliang Wu University of Edinburgh, Ajitha Rajan University of Edinburgh
13:20
20m
Paper
Code Question Answering via Task-Adaptive Sequence-to-Sequence Pre-training
Technical Track
Tingrui Yu School of Software, Shanghai Jiao Tong University, Beijun Shen School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Xiaodong Gu Shanghai Jiao Tong University
13:00 - 14:00
Model Checking 1Technical Track / ERA - Early Research Achievements at Room3
Chair(s): Hiroyuki Nakagawa Osaka University
13:00
20m
Paper
Bisection Value Iteration
Technical Track
Jia Lu East China Normal University, Ming Xu East China Normal University
13:20
20m
Paper
Non-Intrusive Annotation-Based Domain-Specific Analysis to Certify Event-B Models Behaviours
Technical Track
Ismail Mendil INPT-ENSEEIHT/IRIT, Peter Rivière INPT-ENSEEIHT / IRIT, University of Toulouse, France, Yamine Ait Ameur IRIT/INPT-ENSEEIHT, Neeraj Singh INPT-ENSEEIHT / IRIT, University of Toulouse, France, Dominique Mery Université de Lorraine, LORIA, Philippe Palanque ICS-IRIT, Université Toulouse III Paul Sabatier
13:40
15m
Paper
Combining Model-Based Testing and Automated Analysis of Behavioural Models using GraphWalker and UPPAAL
ERA - Early Research Achievements
Saurabh Tiwari DA-IICT Gandhinagar, Parv Shah DA-IICT Gandhinagar, India, Eduard Paul Enoiu Mälardalen University
14:30 - 15:40
14:30
15m
Paper
Unified program cross-architecture migration framework model
SEIP - Software Engineering in Practice
Zhou Minhao Hunan University, Zhe Quan Hunan University, Xin Huang AECC Hunan Aviation powerplant Research Institute
14:45
20m
Paper
Two-Stage Patch Synthesis for API Migration from Single API Usage Example
Technical Track
Daisuke Yamaguchi NTT Corporation, Takuya Iwatsuka NTT
15:05
15m
Paper
Visualizing Contributor Code Competency for PyPI Libraries: Preliminary Results
ERA - Early Research Achievements
Indira Febriyanti Universitas Muhammadiyah Surakarta, Raula Gaikovina Kula Nara Institute of Science and Technology, Ruksit Rojpaisarnkit Nara Institute of Science and Technology, Kanchanok Kannee Nara Institute of Science and Technology, Yusuf Sulistyo Nugroho Universitas Muhammadiyah Surakarta, Kenichi Matsumoto Nara Institute of Science and Technology
15:20
15m
Paper
Why Visualize Competitive Code? Preliminary Categories for Jupyter Notebooks
ERA - Early Research Achievements
Tasha Settewong Mahidol University, Natanon Ritta Mahidol University, Raula Gaikovina Kula Nara Institute of Science and Technology, Chaiyong Rakhitwetsagul Mahidol University, Thailand, Thanwadee Sunetnanta Mahidol University, Kenichi Matsumoto Nara Institute of Science and Technology
14:30 - 15:40
Model Checking 2Technical Track at Room2
Chair(s): Hideto Ogawa Hitachi Ltd.
14:30
20m
Paper
SysML Flow Model
Technical Track
Guohuan Ding East China Normal University, Jing Liu East China Normal University
14:50
20m
Paper
Tuning SAT solvers for LTL Model Checking
Technical Track
Anissa Kheireddine LRDE, Etienne Renault LRDE, Souheib Baarir Université Paris Ouest Nanterre La Défense/LIP6
15:10
20m
Paper
Bounded Model Checking of Synchronous Reactive Models in Ptolemy II
Technical Track
Xiaozhen Zhang Dalian University of Technology, Zhaoming Yang Dalian University of Technology, Hui Kong Huawei Technologies Co., Ltd., Weiqiang Kong Dalian University of Technology
14:30 - 15:40
Source Code Analysis 1Technical Track at Room3
Chair(s): Yoshiki Higo Osaka University
14:30
20m
Paper
Toward a Better Alignment Between the Research and Practice of Code Search Engines
Technical Track
Yin Liu Beijing University of Technology, Shuangyi Li Virginia Tech, Eli Tilevich Virginia Tech
14:50
20m
Paper
Multi-Perspective Alignment Mechanism for Code Search
Technical Track
Shun Yang Wuhan University, Bo Cai Wuhan University
15:10
20m
Paper
Automated Generation of Bug Samples Based on Source Code Analysis
Technical Track
Tianming Zheng Shanghai Jiao Tong University, Zhixin Tong Shanghai Jiao Tong University, Yi-Ping You National Chiao Tung University, Yue Wu Shanghai Jiao Tong University

Thu 8 Dec

Displayed time zone: Osaka, Sapporo, Tokyo change

09:00 - 10:00
December 8thKeynotes at Hall
09:00
60m
Keynote
Improving Development of ML-Enabled Systems through Software Architecture
Keynotes
Grace Lewis Carnegie Mellon Software Engineering Institute
13:00 - 14:30
EducationEDU - Software Engineering Education at Room1
Chair(s): Shinsuke Matsumoto Osaka University
13:00
15m
Paper
Virtual Reality for Software Engineering PresentationsBest EDU Paper Award
EDU - Software Engineering Education
Chaiyong Rakhitwetsagul Mahidol University, Thailand, Morakot Choetkiertikul Mahidol University, Thailand, Apirak Hoonlor Mahidol University, Mores Prachyabrued Mahidol University
13:15
15m
Paper
How students design simple programs. An study on the effort and perceptions of the design activity.
EDU - Software Engineering Education
Diego Vallespir Universidad de la República - Pyxis, Silvana Moreno Universidad de la República Uruguay
13:30
15m
Paper
Automatic Programming Assessment System for Computer Science Bridge Course - An Experience Report
EDU - Software Engineering Education
Sebastian Geiss RWTH Aachen University, Tim Jentzsch RWTH Aachen University, Nils Wild RWTH Aachen University, Christian Plewnia RWTH Aachen University
13:45
15m
Paper
Balancing between Creativity and Efficiency in Software Engineering Project Courses
EDU - Software Engineering Education
Ruoqing Wang Technical University of Munich, Snezhina Milusheva Technical University of Munich, Stephan Krusche Technische Universität München
14:00
15m
Paper
Threshold Concepts and Skills in Software Architecture: Instructors' Perspectives
EDU - Software Engineering Education
Usman Nasir Blekinge Institute of Technology, Muhammad Laiq Blekinge Institute of Technology
13:00 - 14:30
Smart ContractTechnical Track / ERA - Early Research Achievements at Room2
Chair(s): Yoshiki Higo Osaka University
13:00
20m
Paper
Grey-box Fuzzing Based on Execution Feedback for EOSIO Smart Contracts
Technical Track
Wenyin Li Hebei university, Meng Wang Hebei university, Bin Yu Xidian University, Yuhang Shi Xidian Univeristy, Mingxin Fu Xidian Univeristy, You Shao Xidian Univeristy
13:20
15m
Paper
SCAR: Smart Contract Alarm Ranking}
ERA - Early Research Achievements
Kien Tran VNU University of Engineering and Technology, Hieu Vo VNU University of Engineering and Technology
13:35
20m
Paper
Data Flow Reduction Based Test Case Generation for Smart Contracts
Technical Track
Shunhui Ji Hohai University, Shaoqing Zhu Hohai University, Pengcheng Zhang Hohai University, Hai Dong RMIT University
13:55
20m
Paper
A Reference Architecture for Blockchain-based Traceability Systems Using Domain-Driven Design and Microservices
Technical Track
Yanze Wang Nanjing University, Shanshan Li Nanjing University, Huikun Liu Nanjing University, He Zhang Nanjing University, Bo Pan Huawei Technologies Co., Ltd.
13:00 - 14:30
Bug ReportTechnical Track at Room3
Chair(s): Deron Liang National Central University
13:00
20m
Paper
Incremental Relational Topic Model for Duplicate Bug Report Detection
Technical Track
Nitsan Amit Hebrew University, Tien N. Nguyen University of Texas at Dallas
13:20
20m
Paper
Multifaceted Hierarchical Report Identification for Non-Functional Bugs in Deep Learning Frameworks
Technical Track
Guoming Long Loughborough University, Tao Chen Loughborough University, Georgina Cosma Loughborough University
13:40
20m
Paper
Automatic Generation of Metamorphic Relations for a Cyber-Physical System-of-Systems Using Genetic Algorithm
Technical Track
Esther Cho Korea Advanced Institute of Science and Technology, Yong-Jun Shin Korea Advanced Institute of Science and Technology (KAIST), Sangwon Hyun Korea Advanced Institute of Science and Technology, Hansu Kim Korea Advanced Institute of Science and Technology, Doo-Hwan Bae Korea Advanced Institute of Science and Technology
14:00
20m
Paper
An approach for hazard analysis of multiple-cooperative systems considering dynamic configuration uncertainty
Technical Track
Sejin Jung Konkuk university, Junbeom Yoo Konkuk University
15:00 - 16:30
Agile, Business process, and Human AspectsERA - Early Research Achievements / Technical Track / SEIP - Software Engineering in Practice at Room1
Chair(s): Shahida Sulaiman Universiti Teknologi Malaysia
15:00
15m
Paper
AMF: A Field to Realize the Values of Agile Manifesto Through Retrospective
ERA - Early Research Achievements
Ikkaku Kawaguchi University of Tsukuba, Yasuhiro Hayase University of Tsukuba, Kyohei Kobayashi Deloitte Tohmatsu Consulting LLC, Chiemi Watanabe Tsukuba University of Technology
15:15
20m
Paper
Quantifying the Similarity of BPMN Processes
Technical Track
Gwen Salaün University of Grenoble Alpes
15:35
15m
Paper
RP2A: Rare Process-Pattern Analysis - Identifying Potential Problem Process-Patterns by Analyzing System Execution Data-Best SEIP Paper Award
SEIP - Software Engineering in Practice
Shinobu Saito NTT, Kenji Takahashi NTT Security, Yasuyuki Hamada NTT Security Holidings, Jan Vermeulen NTT Americas
15:50
20m
Paper
Incorporating Ethics in Software Engineering: Challenges and Opportunities
Technical Track
Anna Mitchell University of St Andrews, Dharini Balasubramaniam University of St Andrews, Jade Fletcher University of St Andrews
16:10
15m
Paper
Preliminary Analysis of the Influence of the Stereotype Threat on Computer Programming
ERA - Early Research Achievements
Yuriko Takatsuka Kindai University, Masateru Tsunoda Kindai University
15:00 - 16:30
Empirical Studies 2Technical Track at Room2
Chair(s): Yusuf Sulistyo Nugroho Universitas Muhammadiyah Surakarta
15:00
20m
Paper
Exploring Activity and Contributors on GitHub: Who, What, When, and Where
Technical Track
Xiaoya Xia East China Normal University, Zhenjie Weng East China Normal University, will wang , Shengyu Zhao Tongji University
15:20
20m
Paper
The Language of Programming: On the Vocabulary of Names
Technical Track
Nitsan Amit Hebrew University, Dror Feitelson Hebrew University
15:40
20m
Paper
An Empirical Study of Predicting Fault-prone Components and their Evolution
Technical Track
Aparna Pisolkar Gannon University, Md Tajmilur Rahman Gannon University
16:00
20m
Paper
Empirical Study of Co-Renamed Identifiers
Technical Track
Yuki Osumi Tokyo Institute of Technology, Naotaka Umekawa Tokyo Institute of Technology, Hitomi Komata Tokyo Institute of Technology, Shinpei Hayashi Tokyo Institute of Technology
DOI Pre-print
15:00 - 16:30
Machine Learning 2Technical Track at Room3
Chair(s): Morakot Choetkiertikul Mahidol University, Thailand
15:00
20m
Paper
Retrieve-Guided Commit Message Generation with Semantic Similarity And Disparity
Technical Track
Zhihan Li School of Computer Science and Engineering, Central South University, Yi Cheng School of Computer Science and Engineering, Central South University, Haiyang Yang School of Computer Science and Engineering, Central South University, Li Kuang School of Computer Science and Engineering, Central South University, Lingyan Zhang School of Computer Science and Engineering, Central South University
15:20
20m
Paper
Systematic Analysis of Defect Specific Code Abstraction for Neural Program Repair
Technical Track
Kicheol Kim Sungkyunkwan University, Misoo Kim Sungkyunkwan University, Eunseok Lee Sungkyunkwan University
15:40
20m
Paper
NEGAR: Network Embedding Guided Architecture Recovery for Software Systems
Technical Track
Jiayi Chen State Key Lab for Novel Software Technology, Nanjing University, Zhixing Wang State Key Lab for Novel Software Technology, Nanjing University, yuchen jiang , Tian Zhang Nanjing University, Jun Pang University of Luxembourg, Minxue Pan Nanjing University, Nitsan Amit Hebrew University
16:00
20m
Paper
Goal-oriented Knowledge Reuse via Curriculum Evolution for Reinforcement Learning-based Adaptation
Technical Track
Jialong Li Waseda University, Japan, Mingyue Zhang Peking University, China, Zhenyu Mao Waseda University, Haiyan Zhao Peking University, Zhi Jin Peking University, Shinichi Honiden Waseda University / National Institute of Informatics, Japan, Kenji Tei Waseda University

Fri 9 Dec

Displayed time zone: Osaka, Sapporo, Tokyo change

09:20 - 10:30
Review and Natural Language ProcessingTechnical Track / ERA - Early Research Achievements at Room1
Chair(s): Renuka Sindhgatta IBM Research AI
09:20
15m
Paper
A checklist-based approach to assess the systematicity of the abstracts of reviews self-identifying as systematic reviews
ERA - Early Research Achievements
Alvine Boaye Belle York University, Yixi Zhao York University
09:35
15m
Paper
Preliminary Analysis of Review Method Selection Based on Bandit Algorithms
ERA - Early Research Achievements
Takuto Kudo Kindai University, Masateru Tsunoda Kindai University, Amjed Tahir Massey University, Kwabena Ebo Bennin Wageningen University and Research, Akito Monden Okayama University, Koji Toda Fukuoka Institute of Technology, Keitaro Nakasai National Institute of Technology, Kagoshima College, Kenichi Matsumoto Nara Institute of Science and Technology
09:50
20m
Paper
Reducing Syntactic Complexity for Information Extraction from Japanese Requirement Specifications
Technical Track
Maiko Onishi Ochanomizu University, Shinpei Ogata Shinshu University, Kozo Okano Shinshu University, Daisuke Bekki Ochanomizu University
10:10
15m
Paper
Quality assurance study with mismatched data in sentiment analysis
ERA - Early Research Achievements
Tinghui Ouyang National Institute of Advanced Industrial Science and Technology, Yoshiki Seo National Institute of Advanced Industrial Science and Technology, Yutaka Oiwa National Institute of Advanced Industrial Science and Technology
09:20 - 10:30
Microservices and WebERA - Early Research Achievements / Technical Track at Room2
Chair(s): Norihiro Yoshida Ritsumeikan University
09:20
15m
Paper
Testing for Event-Driven Microservices Based on Consumer-Driven Contracts and State Models
ERA - Early Research Achievements
Chu-Fei Wu National Taiwan Ocean University, Shang-Pin Ma National Taiwan Ocean University, An-Chi Shau National Taiwan Ocean University, Hang-Wei Yeh National Taiwan Ocean University
09:35
15m
Paper
Analyzing and Monitoring Kubernetes Microservices based on Distributed Tracing and Service Mesh
ERA - Early Research Achievements
Yu-Te Wang National Taiwan Ocean University, Shang-Pin Ma National Taiwan Ocean University, Yue-Jun Lai National Taiwan Ocean University, Yan-Cih Liang National Taiwan Ocean University
09:50
15m
Paper
Managing Complexity in Microservices Architecture: A Nested MultiTree Domain-Driven Approach
ERA - Early Research Achievements
Duc Minh Le Swinburne Vietnam, FPT University
10:05
20m
Paper
UTCache: Updatable Transactional Cache For Database-Driven Applications
Technical Track
Xiaodong Zhang Shanghai Jiao Tong University, Jing Zhou Shanghai Jiao Tong University
09:20 - 10:30
09:20
15m
Paper
On the Usefulness of Crossover in Search-Based Test Case Generation: An Industrial Report
SEIP - Software Engineering in Practice
Changze Huang Ant Group, Hailian Zhou Ant Group, Hongbing Zhao Ant Group, Wenting Cai Ant Group, Zhi Quan (George) Zhou University of Wollongong, Australia, Mingyue Jiang Zhejiang Sci-Tech University
09:35
15m
Paper
eBAT: An Efficient Automated Web Application Testing Approach Based on Tester’s Behavior
ERA - Early Research Achievements
Fuad Mridha University of Dhaka, Kazi Sakib Institute of Information Technology, University of Dhaka
09:50
20m
Paper
NumFuzz: A Floating-Point Format Aware Fuzzer for Numerical Programs
Technical Track
Chenghu Ma National University of Defense Technology, Liqian Chen National University of Defense Technology, China, Xin Yi National University of Defense Technology, Guangsheng Fan National University of Defense Technology, Ji Wang College of Computer, National University of Defense Technology
10:10
20m
Paper
Detecting High Floating-Point Errors via Ranking Analysis
Technical Track
Zheng Wang National University of Defense Technology, Xin Yi National University of Defense Technology, Hengbiao Yu National University of Defense Technology, Banghu Yin National University of Defense Technology
11:00 - 12:00
December 9thKeynotes at Hall
11:00
60m
Keynote
On the Search for Effective Metamorphic Relations: Overview, Challenges and Opportunities
Keynotes
Shing-Chi Cheung Hong Kong University of Science and Technology
13:00 - 14:00
Source Code Analysis 2Technical Track / ERA - Early Research Achievements at Room1
Chair(s): Yoshiki Higo Osaka University
13:00
20m
Paper
Diff Feature Matching Network in Refactoring DetectionBest Paper Award
Technical Track
Tan Liang , Christoph Bockisch Philipps-Universität Marburg
13:20
15m
Paper
Reusing My Own Code: Preliminary Results for Competitive Coding in Jupyter Notebooks
ERA - Early Research Achievements
Natanon Ritta Mahidol University, Tasha Settewong Mahidol University, Raula Gaikovina Kula Nara Institute of Science and Technology, Chaiyong Rakhitwetsagul Mahidol University, Thailand, Thanwadee Sunetnanta Mahidol University, Kenichi Matsumoto Nara Institute of Science and Technology
13:35
20m
Paper
An Experimental Comparison of Clone Detection Techniques using Java Bytecode
Technical Track
Jean-Guy Schneider Monash University, Sung Une (Sunny) Lee CSIRO's Data61
13:00 - 14:00
Machine Learning 3Technical Track at Room2
Chair(s): Atul Gupta Indian Institute of Information Technology, Design and Manufacturing (IIITDM)
13:00
20m
Paper
Efficient Reinforcement Learning with Generalized-Reactivity Specifications
Technical Track
Chenyang Zhu , Yujie Cai Changzhou University, Can Hu changzhou university, Jia Bi University of Southampton
13:20
20m
Paper
Adversarial Deep Reinforcement Learning for Improving the Robustness of Multi-agent Autonomous Driving Policies
Technical Track
Aizaz Sharif Simula Research Laboratory, Dusica Marijan Simula
13:40
20m
Paper
DronLomaly: Runtime Detection of Anomalous Drone Behaviors via Log Analysis and Deep Learning
Technical Track
Lwin Khin Shar Singapore Management University, Wei Minn Singapore Management University, Duong Ta Singapore Management University, Jiani Fan Nanyang Technological University, Lingxiao Jiang Singapore Management University, Daniel Lim Wai Kiat Singapore Management University
13:00 - 14:00
Symbolic ExecutionTechnical Track at Room3
Chair(s): Shin Hong Handong Global University
13:00
20m
Paper
Symbolic Execution of Floating-point Programs: How far are we?
Technical Track
Guofeng Zhang National University of Defense Technology, Zhenbang Chen National University of Defense Technology, Ziqi Shuai National University of Defense Technology
13:20
20m
Paper
Optimal Refinement-based Array Constraint Solving for Symbolic Execution
Technical Track
Meixi Liu National University of Defense Technology, Changsha, China, Ziqi Shuai National University of Defense Technology, Luyao Liu National University of Defense Technology, Kelin Ma National University of Defense Technology, Ke Ma
13:40
20m
Paper
Synergizing Symbolic Execution and Fuzzing By Function-level Selective Symbolization
Technical Track
Guofeng Zhang National University of Defense Technology, Zhenbang Chen National University of Defense Technology, Ziqi Shuai National University of Defense Technology, Yufeng Zhang Hunan University, Ji Wang College of Computer, National University of Defense Technology

Machine Learning Systems Engineering: Retrospective of Five-Year Activities in Japan

Dr. Hiroshi Maruyama


Abstract

The process of developing machine learning (ML)-based systems is in many aspects different from the process of developing conventional software systems, where the accumulated knowledge on Software Engineering (SE) can guide. In order to facilitate the discussions on what existing SE practices can be applied and what are missing for ML-based systems, we launched a community on Machine Learning Systems Engineering in 2017. Our activities cover a wide range of aspects regarding developing and operating ML-based systems, from requirement development, testing and quality assurance, tools and computing infrastructure, to operating and management issues. This talk reflects on the five year activities, reviews the original goals and what we have achieved and what we have not, and discuss the future directions.

Short Biography

Dr. Hiroshi Maruyama has spent 26 years in IBM Research, Tokyo Research Laboratory, working on various computer science areas such as artificial intelligence, natural language processing, machine translation, hand-writing recognition, multimedia, XML, Web Services, and security. He was the director of IBM Tokyo Research Laboratory from 2006 to 2009. From 2011 to 2016, he was a professor at the Institute of Statistical Mathematics where he worked on projects related to big data, statistics, and their impacts on society. He joined Preferred Networks, Inc. in April 2016 as the chief strategy officer. His current research interests include practical applications of machine learning, social implications of information technology and machine learning, and computer science and statistics in general. Currently he is an Executive Fellow at Kao Corporation, a PFN Fellow at Preferred Networks, and a project professor at the Research into Artifacts, Center for Engineering at the University of Tokyo.

Improving Development of ML-Enabled Systems through Software Architecture

Dr. Grace A. Lewis


Abstract

Developing software systems that contain machine learning (ML) components (ML-enabled systems, or ML systems for short) requires an end-to-end perspective that considers the unique life cycle of these components — from data acquisition, to model training, to model deployment and evolution. However, a problem is that ML system development typically split into three roles, with three different and often separate workflows: data scientists build models; software engineers integrate models into ML systems; and then operations staff deploy, operate, and monitor the ML systems. Because these roles operate separately and often speak different languages, there are opportunities for mismatch between the assumptions made by each role with respect to the elements of the ML-enabled system, and the actual guarantees provided by each element, which leads to system failure. While simply better collaboration between teams is a valid solution to this problem, using software architecture design as the set of activities and artifacts that promote and record collaborative decision making is a much stronger and sustainable solution.

In this talk I will first share the outcomes of two practitioner studies that highlight the problems related to treating ML systems development as a model-centric instead of a system-centric activity. I will then present a set of software architecture practices that can lead to successful ML systems. I will close the talk with some thoughts on remaining gaps in both research and practice that I hope will inspire current and future software engineering and software architecture research.

Short Biography

Grace Lewis is a Principal Researcher at the Carnegie Mellon Software Engineering Institute (SEI) where she conducts applied research on how software engineering and software architecture principles, practices and tools need to evolve in the face of emerging technologies.

She is the principal investigator for the Automating Mismatch Detection and Testing in Machine Learning Systems project that is developing toolsets to support these two activities, in addition to other projects that are advancing the state of the practice in software engineering for machine learning (SE4ML). Grace is also the lead for the Tactical and AI-Enabled Systems (TAS) applied research and development team at the SEI that is creating and transitioning innovative solutions, principles, and best practices for

  • architecting and developing systems to support teams operating at the tactical edge in resource-constrained environments
  • engineering AI software systems
  • using AI/ML at the edge for improved capabilities and mission support

She is currently VP of the IEEE Computer Society Technical & Conference Activities Board (T&C), Diversity and Inclusion (D&I) Vice-Chair for the IEEE Computer Society Technical Community on Software Engineering (TCSE), Alternate Representative for IEEE-CS on the ABET CSAB Board of Directors, as well as an ABET Evaluator for Computer Science undergraduate programs. Grace holds a B.Sc. in Software Systems Engineering and a Post-Graduate Specialization in Business Administration from Icesi University in Cali, Colombia; a Master in Software Engineering from Carnegie Mellon University; and a Ph.D. in Computer Science from Vrije Universiteit Amsterdam.

On the Search for Effective Metamorphic Relations: Overview, Challenges and Opportunities

Prof. Shing-Chi Cheung


Abstract

A major challenge in testing software like artificial intelligent, data-centric, and service-oriented software is the test oracle problem, which occurs when the expected output of a given program input is hard to determine. Instead of examining individual test results, Metamorphic Testing leverages domain-specific relations, called Metamorphic Relations, to address the test oracle problem via a series of test executions. Metamorphic testing is reported to be one of the most popular testing techniques in many application domains such as artificial intelligence, bioinformatics, and search engines. A recent literature review finds that metamorphic testing has been most extensively utilized to test web services, computer graphics, and simulation, making up 40% of related publications. It is also reported to be the most popular testing technique for AI systems. Metamorphic testing has received wide interest from the community. Over 800 research articles on "metamorphic testing" published since 2021 are indexed by Google Scholar. In this talk, we review the key concepts in the deployment of metamorphic testing. We demonstrate the design of effective metamorphic relations for service-oriented software using an example of synchronization bug detection in decentralized applications (DApps) on blockchain. We discuss the major challenges and various research opportunities.

Short Biography

Shing-Chi Cheung received his doctoral degree in Computing from the Imperial College London. After that, he joined the Hong Kong University of Science and Technology (HKUST), where he is a professor of Computer Science and Engineering. He founded the CASTLE research group at HKUST and co-founded in 2006 the International Workshop on Automation of Software Testing (AST), which is now an annual IEEE international conference. He was the General Chair of the 22nd ACM SIGSOFT International Symposium on the Foundations of Software Engineering (FSE 2014), the General Co-chair of APSEC 2012, and the Program Co-chair of APSEC 1996 and 1997. He was an editorial board member of the IEEE Transactions on Software Engineering (TSE, 2006-9). His research interests focus on the testing and analysis for applications on mobile, web, deep learning, open-source repositories, and blockchains. He is an ACM distinguished member. More information about his CASTLE research group can be found at here.