Write a Blog >>

Sliding-window aggregation is a widely-used approach for extracting insights from the most recent portion of a data stream. The aggregations of interest can usually be cast as binary operators that are associative, but they are not necessarily commutative nor invertible. Non-invertible operators, however, are difficult to support efficiently. The best published algorithms require O (log n) aggregation steps per window operation, where n is the sliding-window size at that point. For a FIFO window, this can be improved to O (1) on average by using two aggregation stacks. This paper presents DABA, a novel algorithm for aggregating FIFO sliding windows that significantly improves upon these time bounds. DABA requires only O (1) aggregation steps per operation in the worst case (not just on average). As such, DABA asymptotically improves the performance of sliding-window aggregation without restricting the operator to be invertible. Our experimental results demonstrate that these theoretical improvements hold in practice. DABA is a substantial improvement over the state of the art in terms of both latency and throughput.

Wed 21 Jun

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

11:00 - 12:30
Session 2: High Performance and DistributionDEBS Research Papers at Sala d'Actes, Vertex Building
Chair(s): Guido Salvaneschi TU Darmstadt
11:00
25m
Talk
Minimizing Communication Overhead in Window-Based Parallel Complex Event Processing. (Research Paper)
DEBS Research Papers
Ruben Mayer University of Stuttgart, Muhammad Adnan Tariq University of Stuttgart, Kurt Rothermel Universitaet Stuttgart
11:25
25m
Talk
Low-Latency Sliding-Window Aggregation in Worst-Case Constant Time. (Research Paper)
DEBS Research Papers
Kanat Tangwongsan Mahidol University International College, Martin Hirzel IBM Research, Scott Schneider IBM Research
11:50
20m
Talk
Hardware Accelerated Application Integration Processing. (Industry Paper)
DEBS Research Papers
Daniel Ritter SAP SE, Jonas Dann SAP SE, Norman May SAP SE, Stefanie Rinderle-Ma University of Vienna
12:10
20m
Talk
Chronograph—A Distributed Processing Platform for Online and Batch Computations on Event-sourced Graphs. (Experience Paper)
DEBS Research Papers
Benjamin Erb Ulm University, Germany , Dominik Meißner Institute of Distributed Systems, Ulm University, Jakob Pietron Ulm University, Frank Kargl Ulm University