Metamorphic Testing for Image-based Calcium Imaging Analysis Pipelines
The BRAIN platform that we developed over the last 2.5 years is a collaborative platform for the neuroscience research community, in particular researchers using the emerging methods of optical imaging of neural activity. The BRAIN platform allows neuroscience researchers working on optical imaging to share and manage code, data, and pipelines. Such platforms are crucial for expediting research in data-heavy research communities such as the neuroscience optical imaging community. However, it is important to also build-in quality assurance within the platform ensuring that the results have high quality and can be trusted. Metamorphic testing has been shown to be a powerful technique for scientific data analysis applications. Our idea is to integrate metamorphic testing into the BRAIN platform to automate quality assurance of the components added to the platform as well as of the pipelines themselves. In this paper we report on a feasibility study where we applied metamorphic testing to one step in the calcium imaging pipeline analysis, namely the step to extract the locations of the neuron cells. We tested the cell registration module of CaImAn, one popular software library for calcium imaging analysis, for robust-ness against slight changes in the input images. We applied linear transformations such as small degrees of rotation and blurring to the input images, and then evaluated the neurons identified by the cell registration module for both the original and the transformed inputs. Our results show that for transformations such as rotation and translation, CaImAn’s cell registration unexpectedly identified more neurons for the transformed input than for the original image. Meanwhile, for transformations such as padding, CaImAn identified fewer neurons when run on the the transformed inputs. Our results demonstrates that metamorphic testing is a viable quality assurance technique for the BRAIN platform as well as for similar scientific applications where it is difficult to detect analysis inaccuracies.
Wed 2 JunDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
16:10 - 17:30 | Session 3: Safety and Security and SimulationMET 2021 at MET Room Chair(s): Xiaoyuan Xie School of Computer Science, Wuhan University, China | ||
16:10 30mLong-paper | Metamorphic Detection of Repackaged Malware MET 2021 Media Attached | ||
16:40 20mShort-paper | Metamorphic Testing on the Continuum of Verification and Validation of Simulation Models MET 2021 Media Attached | ||
17:00 30mLong-paper | Metamorphic Testing for Image-based Calcium Imaging Analysis Pipelines MET 2021 Madeline Diep Fraunhofer USA CESE, Rohan Reddy Mekala Fraunhofer USA CESE, Ujjwal Ayyangar Fraunhofer USA CESE, Gudjon Magnusson Fraunhofer USA CESE, Adam Porter Fraunhofer USA CESE, Wolfgang Losert University of Maryland, Mikael Lindvall Fraunhofer USA CESE Media Attached |
Go directly to this room on Clowdr