Commercial and open source database systems consist of millions of lines of highly optimized C code. Yet, their performance on individual queries falls 10x or 100x short of what a hand-written, specialized, implementation of the same query can achieve. In this talk we will present Flare, a SQL query engine implemented entirely in Scala. Where other systems interpret query plans, operator by operator, Flare generates and compiles low-level C code for whole queries at runtime, using the LMS (Lightweight Modular Staging) framework. This approach yields performance up to 100x faster than highly-tuned commercial systems, and compact and high-level code (about ~3000 lines) at the same time. We will also discuss how Flare can be integrated as accelerator back-end with the popular Apache Spark big data platform.

Tiark Rompf is an assistant professor at Purdue University. His work focuses on runtime code generation, advanced compiler technology, and associated language support. From 2008 to 2014 he was a member of Martin Odersky’s Scala team at EPFL where he developed the LMS compiler framework and made various contributions to the Scala language and toolchain (delimited continuations, efficient immutable data structures, compiler speedups, type system work).