Data Lakes Keep Rising While Hadoop Sinks
This may seem contradictory at first glance: Fresh data from the database user community finds that data lakes continue to increase within the enterprise space as big data flows get even bigger. Yet, at the same time, enterprises appear to have pulled back on Hadoop implementations. In earlier days, Hadoop would have been seen as the supporting framework for data lakes. Now, there are many choices emerging for data managers to maintain their data lakes, especially from a plethora of cloud services. If anything, these two opposing trends point to a growing diversity of platforms and approaches now being used to move data as quickly and efficiently as possible from source to scorecard.
New developments in the data world—from cognitive computing to the Internet of Things—are making it critical to take lots of data feeds, pull the points that are of material importance, and engage with them in real time. There are a variety of tools, platforms, and frameworks now available to enterprises to better manage their data. In March 2018, Unisphere fielded a study among DBTA readers to explore the role of new technology initiatives in managing and making this data actionable for the business. This study, sponsored by Oracle, gathered the views and experiences of 203 IT decision makers, representing a broad sample of company types and sizes.
Next-generation data technology initiatives explored in the survey include data lakes, machine learning, Hadoop, Spark, object storage, and the platforms and environments that are supporting them. These distinct technologies are interacting with each other, converging, and paving the way to data-driven enterprises. The survey identified the following five key trends shaping the way enterprises leverage their data, as well as the evolving priorities of data managers.
Data lakes—a place to store diverse datasets without having to build a model first—are perhaps the most mature technology initiatives seen among enterprises in the survey. Adoption of data lakes continues to rise as data managers seek to develop ways to rapidly capture and store data from a multitude of sources in various formats.
