Continuous Analytics on Graph Data Streams Using WSO2 CEP

Is iPaaS solving the right problems? Not knowing the fundamental difference between iPaaS and dPaaS could cost you down the road. Brought to you in partnership with Liaison Technologies.

ACM Distributed and Event-based Systems (DEBS) Grand Challenge is a yearly competition where the participants develop an event-based solution to solve a real world high-volume streaming problem. This year’s challenge focuses on analyzing the properties of a time evolving social-network. The data for this year’s event has been generated using Linked Data Benchmark Council (LDBC) social network data generator.

The ranking of the solutions is carried out by measuring their performance using two performance metrics: (1) throughput and (2) average latency.

WSO2’s been submitting solutions to the grand challenge since 2013, and our previous grand challenge solutions have been ranked as one of the top solutions among the submissions. This year, too, we submitted a solution using WSO2 CEP/Siddhi. Based on its performance, this year’s solution has also been selected as of the top solutions and we were invited to submit a full paper to the DEBS 2016 conference. This paper was presented in the DEBS conference on 22 June 2016.

In this blog I’ll present some details of our solution which is based on WSO2 CEP.

This year’s challenge involves analyzing a dynamic (evolving) social-network graph. Specifically, the 2016 Grand Challenge targets following problems: (1) identification of the posts that currently trigger the most activity in the social network, and (2) identification of large communities that are currently involved in a topic. A brief description of the two queries is given below (more details about the two queries can be found here.