FlowDB: Integrating Stream Processing and Consistent State Management. (Research Paper)
Recent advances in stream processing technologies led to their adoption in many large companies, where they are becoming a core element in the data processing stack. In these settings, stream processors are often used in combination with various kinds of data management frameworks to build software architectures that combine data storage, processing, retrieval, and mining. However, the adoption of separate and heterogeneous subsystems makes these architectures overmuch complex, and this hinders the design, development, maintenance, and evolution of the overall system. We address this issue by proposing a new model that integrates data management within a distributed stream processor. The model enables individual stream processing operators to persist data and make it visible and queryable from external components. It offers flexible mechanisms to control the consistency of data, including transactional updates plus ordering and integrity constraints. The paper contributes to the research on stream processing in various ways: we introduce a new model that has the potential to simplify complex data-intensive applications by integrating data management capabilities within a stream processing system; we define data consistency guarantees and show how they are enforced within this new model; we implement the model into the FlowDB prototype, and study its overhead with respect to a pure stream processing system using real world case studies and synthetic workloads. Finally, we further prove the benefits of the proposed model by showing that FlowDB can outperform a state-of-the-art, in-memory distributed database in data management tasks.
Thu 22 JunDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
13:40 - 15:20 | Session 4: Reliability and SecurityDEBS Research Papers at Sala d'Actes, Vertex Building Chair(s): Roy Friedman Technion - Israel institute of technology | ||
13:40 25mTalk | Maximizing Determinism in Stream Processing Under Latency Constraints. (Research Paper) DEBS Research Papers Nikos Zacheilas Athens University of Economics and Business, Vana Kalogeraki Athens University of Economics and Business, Vincenzo Gulisano Chalmers University of Technology, Yiannis Nikolakopoulos Chalmers University of Technology, Marina Papatriantafilou Chalmers University of Technology, Philippas Tsigas Chalmers University of Technology | ||
14:05 25mTalk | SecureStreams: A Reactive Middleware Framework for Secure Data Stream Processing. (Research Paper) DEBS Research Papers Aurélien Havet Université de Neuchâtel, Rafael Pires Université de Neuchâtel, Pascal Felber Université de Neuchâtel, Marcelo Pasin Université de Neuchâtel, Romain Rouvoy University Lille 1 and INRIA, Valerio Schiavoni University of Neuchatel | ||
14:30 25mTalk | FlowDB: Integrating Stream Processing and Consistent State Management. (Research Paper) DEBS Research Papers Lorenzo Affetti Politecnico di Milano, Alessandro Margara Politecnico di Milano, Gianpaolo Cugola Politecnico di Milano | ||
14:55 25mTalk | A Prototype for Credit Card Fraud Management. (Industry Paper) DEBS Research Papers Alexander Artikis University of Pireaus and NCSR "Demokritos", Nikos Katzouris NCSR Demokritos, Ivo Correia Feedzai, Chris Baber The University of Birmingham, Natan Morar The University of Birmingham, Inna Skarbovsky IBM Haifa Research Lab, Fabiana Fournier IBM Haifa Research Lab, Georgios Paliouras Institute of Informatics & Telecommunications, NCSR "Demokritos" |