Machine learning (ML) models keep getting larger and more complex.
Whereas before models used to be represented by static data-flow graphs, they
are now implemented via arbitrary Python code.
Eager-mode frameworks, such as PyTorch, are now the standard
for developing new ML models.
The semantics of eager-mode frameworks is that operations are computed straight
away.
This greatly simplifies the development process, and it enables more dynamic
ML models.
Although eager-mode frameworks are more convenient, they are less efficient
today as operations are dispatched to the hardware one at a time.
This execution model precludes, for example, operation fusion, which
is essential for executing ML workloads efficiently.
In this paper we present Torchy, a tracing JIT compiler for PyTorch.
Torchy achieves similar performance as data-flow frameworks, while providing
the same semantics of straight-away execution.
Moreover, Torchy works with any PyTorch program unmodified.
Torchy outperforms PyTorch by up to 12x in microbenchmarks, and PyTorch's
static compiler (TorchScript) by up to 5x.
Sat 25 FebDisplayed time zone: Eastern Time (US & Canada) change
14:20 - 15:20 | |||
14:20 20mTalk | A Sound and Complete Algorithm for Code Generation in Distance-Based ISA Research Papers Shu Sugita University of Tokyo, Toru Koizumi University of Tokyo, Ryota Shioya University of Tokyo, Hidetsugu Irie University of Tokyo, Shuichi Sakai University of Tokyo DOI | ||
14:40 20mTalk | Matching Linear Algebra and Tensor Code to Specialized Hardware Accelerators Research Papers Pablo Antonio Martínez University of Murcia, Jackson Woodruff University of Edinburgh, Jordi Armengol-Estapé University of Edinburgh, Gregorio Bernabé University of Murcia, José Manuel García University of Murcia, Michael F. P. O'Boyle University of Edinburgh DOI | ||
15:00 20mTalk | Torchy: A Tracing JIT Compiler for PyTorch Research Papers Nuno P. Lopes INESC-ID; Instituto Superior Técnico - University of Lisbon DOI | ||
