Are Mobile Networks Ready For The Streaming Data Tsunami?
In our latest contribution, Simon Crosby, CTO of SWIM.AI, asks if mobile networks are ready for the forthcoming data deluge, looks at the impact it might have on event processing and how best to handle all that streaming data in real-time
I bet you think this piece is about the rise of Netflix, YouTube, Disney+ etc. It isn’t. Sure, mobile and terrestrial networks are being swamped with video traffic consumed by both young and old alike. But I’m referring to the rise of streaming data from the edge – data from every consumer and industrial product imaginable.
By many estimates over 20 billion smart devices enter the market each year (that’s 2 million per hour), and they all have something to say. A lot to say, and all of the time. The stream of data heading into the mobile network from the edge to carriers and to Internet-facing enterprise apps and SaaS vendors is growing at an enormous rate. So in fact, it isn’t a stream, it’s a tsunami that won’t end.
For mobile operators, there are two opportunities that this data deluge presents. The first is to use device data and network status data to gain real-time insights into network performance, handset performance, user experience, network traffic issues and outages to deliver a more robust network and to improve customer satisfaction.
The second is to use their position at the edge to deliver “Edge Cloud” services that help tame the flood of data before it hits cloud service providers. In this scenario, operators will host edge cloud services on computers close to data sources. The opportunity has never been greater than with the introduction of 5G networking – where operators can offer enterprise customers secure, private slices of network capacity with access to real-time edge computing capabilities with low latency, enabling them to deliver smart cities, smart grids, and tailored enterprise-focused offerings. Vendors have spotted this opportunity too: Ericsson “Edge Gravity” is one example.
What’s needed to succeed across these two areas is for operators to both become fluent in the language of cloud-native messaging services such as those offered by AWS Kinesis, Azure Event Hubs & Enterprise Service Bus, and Google pub/sub, Apache Kafka & Pulsar, Spark etc, and become fluent in open source platforms for real-time stream analysis.
The key to deriving insights from the edge may be in supporting pub/sub messaging. One can argue that pub/sub messaging is a new “dial tone” for both consumer and enterprise-focused service providers. Delivering a platform that helps companies securely scale messaging from edge devices is an important service offering, and just as importantly, adopting cloud-native software architectures is crucial for operators to master in order to deliver customer service and understand the state of their networks, in real-time.
Pub/sub messaging enables an unknown number of publishers to deliver asynchronous messages to subscribers – which are the applications that process it – without either of them needing to know the identity of the other. In pub/sub, sources publish events for a topic to a broker that stores them in the order in which they are received. An application subscribes to one or more topics and the broker forwards matching events.
