Write a Blog >>

Recent advances in stream processing technologies led to their adoption in many large companies, where they are becoming a core element in the data processing stack. In these settings, stream processors are often used in combination with various kinds of data management frameworks to build software architectures that combine data storage, processing, retrieval, and mining. However, the adoption of separate and heterogeneous subsystems makes these architectures overmuch complex, and this hinders the design, development, maintenance, and evolution of the overall system. We address this issue by proposing a new model that integrates data management within a distributed stream processor. The model enables individual stream processing operators to persist data and make it visible and queryable from external components. It offers flexible mechanisms to control the consistency of data, including transactional updates plus ordering and integrity constraints. The paper contributes to the research on stream processing in various ways: we introduce a new model that has the potential to simplify complex data-intensive applications by integrating data management capabilities within a stream processing system; we define data consistency guarantees and show how they are enforced within this new model; we implement the model into the FlowDB prototype, and study its overhead with respect to a pure stream processing system using real world case studies and synthetic workloads. Finally, we further prove the benefits of the proposed model by showing that FlowDB can outperform a state-of-the-art, in-memory distributed database in data management tasks.

Thu 22 Jun
Times are displayed in time zone: (GMT+02:00) Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

debs-2017-papers
13:40 - 15:20: DEBS Research Papers - Session 4: Reliability and Security at Sala d'Actes, Vertex Building
Chair(s): Roy FriedmanTechnion - Israel institute of technology
debs-2017-papers13:40 - 14:05
Talk
Nikos ZacheilasAthens University of Economics and Business, Vana KalogerakiAthens University of Economics and Business, Vincenzo GulisanoChalmers University of Technology, Yiannis NikolakopoulosChalmers University of Technology, Marina PapatriantafilouChalmers University of Technology, Philippas TsigasChalmers University of Technology
debs-2017-papers14:05 - 14:30
Talk
Aurélien HavetUniversité de Neuchâtel, Rafael PiresUniversité de Neuchâtel, Pascal FelberUniversité de Neuchâtel, Marcelo PasinUniversité de Neuchâtel, Romain RouvoyUniversity Lille 1 and INRIA, Valerio SchiavoniUniversity of Neuchatel
debs-2017-papers14:30 - 14:55
Talk
Lorenzo AffettiPolitecnico di Milano, Alessandro MargaraPolitecnico di Milano, Gianpaolo CugolaPolitecnico di Milano
debs-2017-papers14:55 - 15:20
Talk
Alexander ArtikisUniversity of Pireaus and NCSR "Demokritos", Nikos KatzourisNCSR Demokritos, Ivo CorreiaFeedzai, Chris BaberThe University of Birmingham, Natan MorarThe University of Birmingham, Inna SkarbovskyIBM Haifa Research Lab, Fabiana FournierIBM Haifa Research Lab, Georgios PaliourasInstitute of Informatics & Telecommunications, NCSR "Demokritos"