Write a Blog >>
ASE 2021
Sun 14 - Sat 20 November 2021 Australia

[ROSE REPORT]: The artifact is being submitted as part of ROSE initiative. The ROSE team reads papers looking for examples of reuse of tools as well as data sets, methodology, innovative statistical methods, etc. Also, for papers comparing one algorithm to another (e.g. in optimization, data mining, and theorem proving work), we search for “stepping stone” reuse; i.e. (a) some new paper has surveyed the related work to declare that some other algorithm from another paper is the prior state of the art; (b) the new paper then run that prior method (as a baseline); (c) new results from new algorithms are then compared against the baseline.

If the new paper reuses something from an older paper (e.g. using code, methodology, etc) we see that the claim that the prior thing was useful for some task and see REPRODUCTION of the claim of the original paper. If the new paper implements their own version of the older idea, then that is a REPLICATION of the claim that some prior method is useful for some task. As part of that work, we report here the following example of REPRODUCTION.

We reported here the replication of LIME. In 2016, Ribeiro et al. proposed an approach LIME, which is a novel explanation approach that can explain the prediction outcomes of any machine learning classifier or regressor. They evaluated LIME with simulated and human subjects in the terms of trust and associated tasks. In 2020, Zhao et al. proposed eWarn, an incident prediction tool for online service systems. In eWarn, they replicated LIME to explain a prediction result generated by eWarn.