CC 2023
Sat 25 - Sun 26 February 2023 Montréal, Canada
Sat 25 Feb 2023 15:00 - 15:20 at St. Laurent 3 - Code Generation & Synthesis Chair(s): Mary Hall

Machine learning (ML) models keep getting larger and more complex.
Whereas before models used to be represented by static data-flow graphs, they
are now implemented via arbitrary Python code.
Eager-mode frameworks, such as PyTorch, are now the standard
for developing new ML models.
The semantics of eager-mode frameworks is that operations are computed straight
This greatly simplifies the development process, and it enables more dynamic
ML models.

Although eager-mode frameworks are more convenient, they are less efficient
today as operations are dispatched to the hardware one at a time.
This execution model precludes, for example, operation fusion, which
is essential for executing ML workloads efficiently.

In this paper we present Torchy, a tracing JIT compiler for PyTorch.
Torchy achieves similar performance as data-flow frameworks, while providing
the same semantics of straight-away execution.
Moreover, Torchy works with any PyTorch program unmodified.
Torchy outperforms PyTorch by up to 12x in microbenchmarks, and PyTorch's
static compiler (TorchScript) by up to 5x.

Sat 25 Feb

Displayed time zone: Eastern Time (US & Canada) change

14:20 - 15:20
Code Generation & SynthesisResearch Papers at St. Laurent 3
Chair(s): Mary Hall University of Utah
A Sound and Complete Algorithm for Code Generation in Distance-Based ISA
Research Papers
Shu Sugita University of Tokyo, Toru Koizumi University of Tokyo, Ryota Shioya University of Tokyo, Hidetsugu Irie University of Tokyo, Shuichi Sakai University of Tokyo
Matching Linear Algebra and Tensor Code to Specialized Hardware Accelerators
Research Papers
Pablo Antonio Martínez University of Murcia, Jackson Woodruff University of Edinburgh, Jordi Armengol-Estapé University of Edinburgh, Gregorio Bernabé University of Murcia, José Manuel García University of Murcia, Michael F. P. O'Boyle University of Edinburgh
Torchy: A Tracing JIT Compiler for PyTorch
Research Papers
Nuno P. Lopes INESC-ID; Instituto Superior Técnico - University of Lisbon