Efficiently Learning Locality Optimizations by Decomposing Transformation Domains
Optimizing compilers for efficient machine learning are more important than ever due to the rising ubiquity of the application domain in numerous facets of life. Predictive model-guided compiler optimization is sometimes used to derive sequences of loop transformations that optimize the performance of the machine learning computations. However, training-data generation for these models often requires the traversal of prohibitively expensive schedule spaces and executing code variants corresponding to different schedule options. The size of these search spaces can quickly explode when predicting the combined effects of multiple loop transformations. This paper characterizes a learning strategy for deriving transformation sequences called Composed Singular Prediction (CSP).
Instead of a monolithic cost model that predicts the profitability of a given transformation sequence, CSP exploits a collection of cost models, each trained on a particular loop transformation domain. In a case study, a domain-specific compiler deploys the learned models to predict loop tiling and loop permutation schedules to perform data locality optimization of Conv2d kernels. The system achieves performance improvements up to 4.0x against Intel oneDNN while saving ~105.3x in training data collection time compared to exhaustive exploration of the design space.
Sat 25 FebDisplayed time zone: Eastern Time (US & Canada) change
11:20 - 12:20
|Efficiently Learning Locality Optimizations by Decomposing Transformation Domains|
|A Deep Learning Model for Loop Interchange|
Lina Mezdour NYU Abu Dhabi; ESI, Khadidja Kadem NYU Abu Dhabi; ESI, Massinissa Merouani NYU Abu Dhabi, Amina Selma Haichour ESI, Saman Amarasinghe Massachusetts Institute of Technology, Riyadh Baghdadi NYU Abu DhabiDOI
|(De/Re)-Compositions Expressed Systematically via MDH-Based Schedules|
Ari Rasch University of Muenster, Richard Schulze University of Muenster, Denys Shabalin Google, Anne Elster NTNU, Sergei Gorlatch University of Muenster, Mary Hall University of UtahDOI