7 top trends that will impact data management and cloud computing in 2020

In 2019 we saw the beginnings of the new data organization. Teams dedicated to supporting growing AI and analytic workloads grew and became more common, and as a result the technologies to power those workloads have become more critical.

2020 brings this new data organization to the forefront, and this new organization will focus on investing in the public cloud while maintaining their on-prem Hadoop, one ML framework to rule them all, and “Kubernetifying” the analytics stack to name just a few. These are the trends I believe we’ll see come about in 2020.

Rise of the hybrid cloud (really)

We’ve been hearing people talk about the hybrid cloud for the past three years now. And for the most part, that’s all it’s been – talk. 2020 is the year it gets real. We are seeing large enterprises refusing to add capacity on-prem to their Hadoop deployments and instead invest in the public cloud. But they are still not willing to move their core enterprise data to the cloud. Data will stay on-prem and compute will be burst to the cloud, particularly for peak demands and unpredictable workloads. Technologies that provide optimal approaches to achieve this will drive the rise of the hybrid cloud.

One Machine Learning framework to rule them all

Machine learning with models has reached a turning point, with companies of all sizes and at all stages moving towards operationalizing their model training efforts. While there are several popular frameworks for model training, a leading technology hasn’t yet emerged. Just like Apache Spark is considered a leader for data transformation jobs and Presto is emerging as the leading tech for interactive querying, 2020 will be the year we’ll see a frontrunner dominate the broader model training space with pyTorch or Tensorflow as leading contenders.

While containers and Kubernetes works exceptionally well for stateless applications like web servers and self-contained databases, we haven’t seen a ton of container usage when it comes to advanced analytics and AI. In 2020, we’ll see a shift to AI and analytic workloads becoming more mainstream in Kubernetes land. “Kubernetifying” the analytics stack will mean solving for data sharing and elasticity by moving data from remote data silos into K8s clusters for tighter data locality.

There is a lot of talk about Hadoop being dead…but the Hadoop ecosystem has rising stars.