Write a Blog >>

Sliding-window aggregation is a widely-used approach for extracting insights from the most recent portion of a data stream. The aggregations of interest can usually be cast as binary operators that are associative, but they are not necessarily commutative nor invertible. Non-invertible operators, however, are difficult to support efficiently. The best published algorithms require O (log n) aggregation steps per window operation, where n is the sliding-window size at that point. For a FIFO window, this can be improved to O (1) on average by using two aggregation stacks. This paper presents DABA, a novel algorithm for aggregating FIFO sliding windows that significantly improves upon these time bounds. DABA requires only O (1) aggregation steps per operation in the worst case (not just on average). As such, DABA asymptotically improves the performance of sliding-window aggregation without restricting the operator to be invertible. Our experimental results demonstrate that these theoretical improvements hold in practice. DABA is a substantial improvement over the state of the art in terms of both latency and throughput.

Wed 21 Jun

11:00 - 12:30: DEBS Research Papers - Session 2: High Performance and Distribution at Sala d'Actes, Vertex Building
Chair(s): Guido SalvaneschiTU Darmstadt
debs-2017-papers149803560000011:00 - 11:25
Ruben MayerUniversity of Stuttgart, Muhammad Adnan TariqUniversity of Stuttgart, Kurt RothermelUniversitaet Stuttgart
debs-2017-papers149803710000011:25 - 11:50
Kanat TangwongsanMahidol University International College, Martin HirzelIBM Research, Scott SchneiderIBM Research
debs-2017-papers149803860000011:50 - 12:10
Daniel RitterSAP SE, Jonas DannSAP SE, Norman MaySAP SE, Stefanie Rinderle-MaUniversity of Vienna
debs-2017-papers149803980000012:10 - 12:30
Benjamin ErbUlm University, Germany , Dominik MeißnerInstitute of Distributed Systems, Ulm University, Jakob PietronUlm University, Frank KarglUlm University