Building a Node.js Benchmark: Initial Steps
To address our first goal, we start by downloading meta-data for all projects that use NPM on GitHub, collecting metrics such as: # of dependencies, # open and closed issues, # stars, # topics, size, # commits, # contributors, and proportion of project-specific code vs. dependency code. Then, we infer the probability distribution of each metrics (e.g. normal, log-normal, exponential, etc.) and sample projects in such a way that the original probability distributions are preserved. To address our second goal, we further select projects that have working test suites, pre-download all their dependencies, and build a harness that runs tests from all projects. Then, we build a test environment in the form of a container or virtual machine where specific versions of Node.js, web browsers, and databases are pre-installed, in an attempt to minimize variations across users of our benchmark.
During the talk, we will explain how we collected various metrics from GitHub, how we inferred their distributions, and how we sampled projects. We will also share our experience building an initial, executable, benchmark suite of 50 applications, and highlight the challenges we foresee ex-panding it. Finally, we will discuss two potential uses of the benchmark: dynamic analyses for performance, and security, and the additional metrics that we might need to consider.
|Slides (Building nodejs benchmark.pdf)||803KiB|
Wed 18 JulDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
14:00 - 15:30
Saam Barati AppleFile Attached
Petr Maj Czech Technical University, Celeste Hollenbeck Northeastern University, USA, Shabbir Hussain Northeastern University, Jan Vitek Northeastern University
|Building a Node.js Benchmark: Initial Steps|
Petr Maj Czech Technical University, François Gauthier Oracle Labs, Celeste Hollenbeck Northeastern University, USA, Jan Vitek Northeastern University, Cristina Cifuentes Oracle LabsFile Attached
|A Micro-Benchmark for Dynamic Program Behaviour|
Li Sui Massey University, New Zealand, Jens Dietrich Massey University, Michael Emery Massey University, Amjed Tahir Massey University, Shawn Rasheed Massey University