Edge Analytics: Pros & Cons of Immediate, Local Insight
A number of data scientists reached out to me about data storage and processing as discussed in my last blog around ‘IoT’. All of their questions largely fell into the same bucket: they are puzzled by what to do with their data. Whether they should store or discard their enterprise data, and if stored, what is the best approach they can take to making that data a strategic asset for their company.
Despite widespread proliferation of sensors, the majority of industrial internet of things, or ‘IIoT’ data, collected is never analysed—which is tragic. Many existing IoT platform solutions are painfully slow, expensive and a drain on resources—which makes analysing the rest extremely difficult. Gartner mentioned that 90% of deployed data will be useless and Experian mentioned about 32% of data in US firms to be inaccurate. The key takeaway is that data is the most valuable asset for any company. So it would be a shame to completely discard or let it lie dormant in an abandoned data lake somewhere. It’s imperative that all data scientists tap into their swelling pools of IoT data to make sense of the various endpoints of information and help develop conclusions that will ultimately deliver business outcomes. I am totally against of discarding data without processing.
As mentioned in ‘IoT Blog’, in few years there will be an additional 15 to 40 billion devices generating data from the edge vs. what we have today[1]. That brings new challenges. Just imagine an infrastructure transferring this data to data lakes and processing hubs to process. The load will continue to rise exponentially over coming months and years, creating just another problem of stretching the limits of your infrastructure. The only benefit of this data will come from analysis either it is traffic of “things” or surveillance cameras. In time critical situations, if we delay this analysis that might be “too late”. The delay could be due to many reasons like limited network availability or overloaded central systems.
A relatively new approach namely “edge analytics” is in use to address these issues. Basically it is as simple as to say, perform analysis at the point where data is being generated. It’s about analysing in real-time on site. The architectural design of “things” should consider built-in analysis. For example, sensors in train or at stop lights that provide intelligent monitoring and management of traffic should be powerful enough to raise an alarm to nearby fire or police departments based on their analysis of the local surroundings. Another good example is security cameras. To transmit the live video without any change is pretty much useless.
