Towards Understanding Machine Learning Testing in Practise
Sat 20 May 2023 15:06 - 15:12 at Meeting Room 105 - Onsite - Poster
Visualisations drive all aspects of the Machine Learning (ML) Development Cycle but remain a vastly untapped resource by the research community. ML testing is a highly interactive and cognitive process which demands a human-inthe- loop approach. Besides writing tests for the code base, bulk of the evaluation requires application of domain expertise to generate and interpret visualisations. To gain a deeper insight into the process of testing ML systems, we propose to study visualisations of ML pipelines by mining Jupyter notebooks. We propose a two prong approach in conducting the analysis. First, gather general insights and trends using a qualitative study of a smaller sample of notebooks. And then use the knowledge gained from the qualitative study to design an empirical study using a larger sample of notebooks. Computational notebooks provide a rich source of information in three formats—text, code and images. We hope to utilise existing work in image analysis and Natural Language Processing for text and code, to analyse the information present in notebooks. We hope to gain a new perspective into program comprehension and debugging in the context of ML testing.
Mon 15 MayDisplayed time zone: Hobart change
20:45 - 22:15 | Poster - OnlinePosters / Papers at Virtual - Zoom for CAIN Chair(s): Mona Rahimi Northern Illinois University, Karthik Vaidhyanathan IIIT Hyderabad Click here to Join us over zoomClick Here to watch the session recording on YouTube | ||
20:45 6mPoster | AI Living Lab: Quality Assurance for AI-based Health systems Posters | ||
20:51 6mPoster | AI Planning Software Development Lifecycle Posters Ilche Georgievski University of Stuttgart, Germany File Attached | ||
20:57 6mPoster | Algorithm Debt: Challenges and Future Paths Posters Emmanuel Iko-Ojo Simon Australian National University, Melina Vidoni Australian National University, Fatemeh Hendijani Fard University of British Columbia | ||
21:03 6mPoster | Enabling Machine Learning in Software Architecture Frameworks Posters Armin Moin University of California, Santa Barbara, Atta Badii University of Reading, United Kingdom, Stephan G¨unnemann School of Computation, Information and Technology, Technical University of Munich, Munich, Germany, Moharram Challenger University of Antwerp DOI Pre-print | ||
21:09 6mPoster | Extensible Modeling Framework for Reliable Machine Learning System Analysis Posters Jati Hiliamsyah Husen Waseda University, Hironori Washizaki Waseda University, Hnin Thandar Tun Waseda University, Japan, Nobukazu Yoshioka Waseda University, Japan, Yoshiaki Fukazawa Waseda University, Hironori Takeuchi Musashi University, Hiroshi Tanaka Fujitsu Limited, Tokyo, Japan, Kazuki Munakata Fujitsu Limited, Tokyo, Japan | ||
21:15 6mPoster | How Federated Machine Learning Helps Increase the Mutual Benefit of Data-Sharing Ecosystems Posters Iva Krasteva Sofia University, GATE Institute, Boris Kraychev GATE Institute, Ensiye Kiyamousavi GATE Institute | ||
21:21 6mPoster | Maintaining and Monitoring AIOps Models Against Concept Drift Posters Lorena Poenaru-Olaru TU Delft, Luís Cruz Delft University of Technology, Jan S. Rellermeyer Leibniz University Hannover, Arie van Deursen Delft University of Technology | ||
21:27 6mPoster | Reproducibility Requires Consolidated Artifacts Posters Iordanis Fostiropoulos University of Southern California, USA, Bowman Brown University of Southern California, USA, Laurent Itti University of Southern California, USA | ||
21:33 6mPoster | Tenet: A Flexible Framework for Machine Learning-based Vulnerability Detection Posters Eduard Costel Pinconschi Instituto Superior Técnico, University of Lisboa & INESC-ID, Sofia Reis Instituto Superior Técnico, U. Lisboa & INESC-ID, Chi Zhang , Rui Abreu Faculty of Engineering, University of Porto, Hakan Erdogmus Carnegie Mellon University, Limin Jia Carnegie Mellon University | ||
21:39 6mPoster | Towards Understanding Machine Learning Testing in Practise Posters Arumoy Shome Delft University of Technology, Luís Cruz Delft University of Technology, Arie van Deursen Delft University of Technology Pre-print | ||
21:45 30mBreak | Break Out Session - Online Papers |
Sat 20 MayDisplayed time zone: Hobart change
15:00 - 15:30 | |||
15:00 6mPoster | Maintaining and Monitoring AIOps Models Against Concept Drift Posters Lorena Poenaru-Olaru TU Delft, Luís Cruz Delft University of Technology, Jan S. Rellermeyer Leibniz University Hannover, Arie van Deursen Delft University of Technology | ||
15:06 6mPoster | Towards Understanding Machine Learning Testing in Practise Posters Arumoy Shome Delft University of Technology, Luís Cruz Delft University of Technology, Arie van Deursen Delft University of Technology Pre-print | ||
15:12 6mPoster | Algorithm Debt: Challenges and Future Paths Posters Emmanuel Iko-Ojo Simon Australian National University, Melina Vidoni Australian National University, Fatemeh Hendijani Fard University of British Columbia | ||
15:18 6mPoster | Extensible Modeling Framework for Reliable Machine Learning System Analysis Posters Jati Hiliamsyah Husen Waseda University, Hironori Washizaki Waseda University, Hnin Thandar Tun Waseda University, Japan, Nobukazu Yoshioka Waseda University, Japan, Yoshiaki Fukazawa Waseda University, Hironori Takeuchi Musashi University, Hiroshi Tanaka Fujitsu Limited, Tokyo, Japan, Kazuki Munakata Fujitsu Limited, Tokyo, Japan | ||
15:24 6mPoster | AI Living Lab: Quality Assurance for AI-based Health systems Posters |