Parsimony: Enabling SIMD/Vector Programming in Standard Compiler Flows
Achieving peak throughput on modern CPUs requires maximizing the use of single-instruction, multiple-data (SIMD) or vector compute units. Single-program, multiple-data (SPMD) programming models are an effective way to use high-level programming languages to target these ISAs. Unfortunately, many SPMD frameworks have evolved to have either overly-restrictive language specifications or under-specified programming models, and this has slowed the widescale adoption of SPMD-style programming. This paper introduces Parsimony (PARallel SIMd), a SPMD programming approach built with semantics designed to be compatible with multiple languages and to cleanly integrate into the standard optimizing compiler toolchains for those languages. We first explain the Parsimony programming model semantics and how they enable a standalone compiler IR-to-IR pass that can perform vectorization independently of other passes, improving the language and toolchain compatibility of SPMD programming. We then demonstrate a LLVM prototype of the Parsimony approach that matches the performance of ispc, a popular but more restrictive SPMD approach, and achieves 97% of the performance of hand-written AVX-512 SIMD intrinsics on over 70 benchmarks ported from the Simd Library. We finally discuss where Parsimony has exposed parts of existing language and compiler flows where slight improvements could further enable improved SPMD program vectorization.
Tue 28 FebDisplayed time zone: Eastern Time (US & Canada) change
15:40 - 17:00 | Session 6 -- Tool and Practical Experience IIMain Conference at Montreal 1-2-3 Chair(s): Ayal Zaks Mobileye | ||
15:40 26mTalk | Bridging Control-Centric and Data-Centric Optimization Main Conference Tal Ben-Nun ETH Zurich, Berke Ates ETH Zurich, Alexandru Calotoiu ETH Zurich, Torsten Hoefler ETH Zurich DOI | ||
16:06 26mTalk | Parsimony: Enabling SIMD/Vector Programming in Standard Compiler Flows Main Conference Vijay Kandiah Northwestern University, Daniel Lustig NVIDIA, Oreste Villa NVIDIA, David Nellans NVIDIA, Nikos Hardavellas Northwestern University DOI | ||
16:33 26mTalk | Program State Element Characterization Main Conference Enrico Armenio Deiana Northwestern University, Brian Suchy Northwestern University, Michael Wilkins Northwestern University, Brian Homerding Northwestern University, Tommy McMichen Northwestern University, Katarzyna Dunajewski Northwestern University, Peter Dinda Northwestern University, Nikos Hardavellas Northwestern University, Simone Campanoni Northwestern University DOI |