Are Data Lakes Just Dumping Grounds?
In this special guest feature Stan Christiaens, Co-founder and CTO at Collibra, shares strategies to ensure that data moves beyond raw material to take its rightful place as a valuable business asset. The article outlines common problems with data lakes, strategies for how business can avoid those problems, and how governance enables a data lake to become more than just a data repository. Stan leads a global product organization with a focus on driving innovation in data governance and catalog software. Prior to co-founding Collibra, Stan was a senior researcher at the Vrije Universiteit of Brussels, a leading semantic research center in Europe, where he focused on application-oriented research in semantics. He holds a Master of Science degree in Information Technology and a Master’s degree in AI from Katholieke Universiteit Leuven and a Postgraduate in Industrial Corporate Governance from Europese Hogeschool Brussel.
Big data. Although the term is ubiquitous today, it wasn’t so long ago that “big data” wasn’t part of the everyday lexicon. The growth of data, in all its forms, during the last five years has been dizzying, and has caught many organizations worldwide flat-footed.
In light of this, finding a way to deal with all of this data has become big business. IDC estimates that worldwide revenue for big data and business analytics will grow to more than $260 billion by 2022. Organizations have made significant investments in hardware, software and services to deal with the onslaught of data.
Data lakes quickly emerged as a technology front-runner in the race to make data more digestible – and to finally get it in one place. Data lakes are flexible, scalable and offer an easy solution to store data. They serve as central repositories for all types of “raw” data, including structured, semi-structured and unstructured. The data structure and requirements aren’t defined until the data is needed. Ideally, a data lake is the go-to location for data scientist and business users alike, fueling all analytics activities across the business.
The reality is that getting insight and value from so much data is challenging. Forrester finds that between 60 – 73% of all enterprise data goes unused for analytics. It’s all too common for the majority of users to only find a small percentage of truly valuable in this wide array of assets. In the rush to aggregate our data somewhere, lakes have become swamps of undefined data from a variety of sources. Data scientists and everyday business users struggle to find and understand data.
