Towards Building AI-CPS with NVIDIA Isaac Sim: An Industrial Benchmark and Case Study for Robotics Manipulation
As a representative cyber-physical system (CPS), robotic manipulator has been widely adopted in various academic research and industrial processes, indicating its potential to act as a universal interface between the cyber and the physical worlds. Recent studies in robotics manipulation have started employing artificial intelligence (AI) approaches as controllers to achieve better adaptability and performance. However, the inherent challenge of explaining AI components introduces uncertainty and unreliability to these AI-enabled robotics systems, necessitating a reliable development platform for system design and performance assessment. As a foundational step towards building reliable AI-enabled robotics systems, in this paper, we propose a public industrial benchmark for robotics manipulation. It leverages NVIDIA Omniverse Isaac Sim as the simulation platform, encompassing eight representative manipulation tasks and multiple AI software controllers. An extensive empirical evaluation is conducted to analyze the performance of AI controllers in solving robotics manipulation tasks, enabling a relatively thorough understanding of their effectiveness. To further demonstrate the applicability of our benchmark, we also develop a first falsification framework that is compatible with Isaac Sim. This framework bridges the gap between traditional falsification methods and modern physics engine-based simulations. The effectiveness of different optimization methods in falsifying AI-enabled robotics manipulation with physical simulators is also examined. Our work not only establishes a foundation for the design and development of AI-enabled robotics systems but also provides practical experience and guidance to practitioners in this field, promoting further research in this critical academic and industrial domain. The benchmarks, source code, and detailed evaluation results are available at https://sites.google.com/view/ai-cps-robotics-manipulation/home.
Fri 19 AprDisplayed time zone: Lisbon change
11:00 - 12:30 | LLM, NN and other AI technologies 5Software Engineering Education and Training / Software Engineering in Practice / Research Track at Grande Auditório Chair(s): Baishakhi Ray AWS AI Labs | ||
11:00 15mTalk | Enhancing Exploratory Testing by Large Language Model and Knowledge Graph Research Track Yanqi Su Australian National University, Dianshu Liao Australian National University, Zhenchang Xing CSIRO's Data61, Qing Huang School of Computer Information Engineering, Jiangxi Normal University, Mulong Xie CSIRO's Data61, Qinghua Lu Data61, CSIRO, Xiwei (Sherry) Xu Data61, CSIRO | ||
11:15 15mTalk | LLMParser: An Exploratory Study on Using Large Language Models for Log Parsing Research Track Zeyang Ma Concordia University, An Ran Chen University of Alberta, Dong Jae Kim Concordia University, Tse-Hsun (Peter) Chen Concordia University, Shaowei Wang Department of Computer Science, University of Manitoba, Canada | ||
11:30 15mTalk | Enhancing Text-to-SQL Translation for Financial System Design Software Engineering in Practice Yewei Song University of Luxembourg, Saad Ezzini Lancaster University, Xunzhu Tang University of Luxembourg, Cedric Lothritz University of Luxembourg, Jacques Klein University of Luxembourg, Tegawendé F. Bissyandé University of Luxembourg, Andrey Boytsov Banque BGL BNP Paribas, Ulrick Ble Banque BGL BNP Paribas, Anne Goujon Banque BGL BNP Paribas | ||
11:45 15mTalk | Towards Building AI-CPS with NVIDIA Isaac Sim: An Industrial Benchmark and Case Study for Robotics Manipulation Software Engineering in Practice Zhehua Zhou University of Alberta, Jiayang Song University of Alberta, Xuan Xie University of Alberta, Zhan Shu University of Alberta, Lei Ma The University of Tokyo & University of Alberta, Dikai Liu NVIDIA AI Tech Centre, Jianxiong Yin NVIDIA AI Tech Centre, Simon See NVIDIA AI Tech Centre Pre-print | ||
12:00 15mTalk | Let's Ask AI About Their Programs: Exploring ChatGPT's Answers To Program Comprehension Questions Software Engineering Education and Training Pre-print Media Attached File Attached | ||
12:15 15mTalk | Experience Report: Identifying common misconceptions and errors of novice programmers with ChatGPT Software Engineering Education and Training Hua Leong Fwa Singapore Management University Media Attached |