Maximizing Determinism in Stream Processing Under Latency Constraints. (Research Paper)
The problem of coping with the demands of determinism and meeting latency constraints is challenging in distributed data stream processing systems that have to process high volume data streams that arrive from different unsynchronized input sources. In order to deterministically process the streaming data, they need mechanisms that synchronize the order in which tuples are processed by the operators. On the other hand, achieving real-time response in such a system requires careful tradeoff between determinism and low latency performance. We build on a recently proposed approach to handle data exchange and synchronization in stream processing, namely ScaleGate, which comes with guarantees for determinism and an efficient lock-free implementation, enabling high scalability. Considering the challenge and trade-offs implied by real-time constraints, we propose a system which comprises (a) a novel data structure called Slack-ScaleGate (SSG), along with its algorithmic implementation; SSG enables us to guarantee the deterministic processing of tuples as long as they are able to meet their latency constraints, and (b) a method to dynamically tune the maximum amount of time that a tuple can wait in the SSG data-structure, relaxing the determinism guarantees when needed, in order to satisfy the latency constraints. Our detailed experimental evaluation using a traffic monitoring application deployed in the city of Dublin, illustrates the working and benefits of our approach.
Thu 22 JunDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
13:40 - 15:20 | Session 4: Reliability and SecurityDEBS Research Papers at Sala d'Actes, Vertex Building Chair(s): Roy Friedman Technion - Israel institute of technology | ||
13:40 25mTalk | Maximizing Determinism in Stream Processing Under Latency Constraints. (Research Paper) DEBS Research Papers Nikos Zacheilas Athens University of Economics and Business, Vana Kalogeraki Athens University of Economics and Business, Vincenzo Gulisano Chalmers University of Technology, Yiannis Nikolakopoulos Chalmers University of Technology, Marina Papatriantafilou Chalmers University of Technology, Philippas Tsigas Chalmers University of Technology | ||
14:05 25mTalk | SecureStreams: A Reactive Middleware Framework for Secure Data Stream Processing. (Research Paper) DEBS Research Papers Aurélien Havet Université de Neuchâtel, Rafael Pires Université de Neuchâtel, Pascal Felber Université de Neuchâtel, Marcelo Pasin Université de Neuchâtel, Romain Rouvoy University Lille 1 and INRIA, Valerio Schiavoni University of Neuchatel | ||
14:30 25mTalk | FlowDB: Integrating Stream Processing and Consistent State Management. (Research Paper) DEBS Research Papers Lorenzo Affetti Politecnico di Milano, Alessandro Margara Politecnico di Milano, Gianpaolo Cugola Politecnico di Milano | ||
14:55 25mTalk | A Prototype for Credit Card Fraud Management. (Industry Paper) DEBS Research Papers Alexander Artikis University of Pireaus and NCSR "Demokritos", Nikos Katzouris NCSR Demokritos, Ivo Correia Feedzai, Chris Baber The University of Birmingham, Natan Morar The University of Birmingham, Inna Skarbovsky IBM Haifa Research Lab, Fabiana Fournier IBM Haifa Research Lab, Georgios Paliouras Institute of Informatics & Telecommunications, NCSR "Demokritos" |