Big Data Platforms in 2017: Leveraging Hybrid Clouds for Intelligent Operations
According to a recent Gartner survey, big data investments reached a possible peak in 2016. But these investments show signs of contracting, with 48 percent of companies having invested in big data in 2016 – only three percent more than 2015. This trend signals that organizations have delivered significant big data insights and will now look to operationalize in the coming year (and beyond).
With the rise of hybrid cloud environments, we expect that companies will operationalize existing big data insights to heighten the efficiency of critical business operations in the year ahead. We’ll see this both in the movement to streamline operations using big data sets (both in the cloud and on the ground), and in the tech advances and standards in 2017 to overcome roadblocks to operationalizing created by these hybrid cloud environments.
In 2016, we used customer management cloud apps to start to explore the benefits of integrating big data insights from enterprises’ on-premises platforms; these were separated by both functional areas and disconnected technology stacks. In 2017, we will continue to see functional areas collaborate with IT, and see improved connectivity with the big data ecosystem to really expand the use of existing big data platforms through increased access to that data, which will be of central importance to customer experiences.
Data lakes have become a valuable repository to store all facets of customer data from a variety of data streams – the different systems used by internal functional departments such as CRM, marketing automation, web analytics, survey platforms, webinar data, and so on. This creates a single repository of insights, forging the foundation for new and advanced analytics techniques using data sets with value that has yet to be derived.
The predicted movement of this data will be driven from customer management cloud applications to access the detailed “big data” data on-demand in the flexible spirit of the data lake (for example, being able to ask questions with an ad hoc schema on-read approach). In contrast, some of the more aggregated data (“not as big data”) will get physically moved to the cloud for more repeatable use cases.
When thinking about the enterprise, many analytics and reporting platforms continue to run on-premises on private cloud or grid infrastructures. Big data volumes continue to grow in the cloud, and it’s not feasible to crunch those cloud resident big data sets in on-premises data centers. This is where cloud big data platforms such as Amazon EMR, IBM BigInsights on Cloud, Microsoft Azure HDInsight, or SAP Altiscale, are often more scalable and cost effective to crunch and transform big data sets into business insights. We have already seen this in 2016, and predict 2017 will be the year in which we will start to integrate those insights, which are manageable in size, by moving them into on-premise databases and analytics platforms for core business operations.
