A Deep Dive Into Data Lakes

Data lakes are centralized storage and data repositories that allow you to work with a variety of different types of data. (Photo: Rich Miller)

In the age of Big Data, we’ve had to come up with new terms to describe large-scale data storage. We have databases, data warehouses and now data lakes.

While they all contain data, these terms describe different ways of storing and using that data. Before we discuss data lakes and why they are important, let’s examine how they differ from databases and data warehouses.

Let’s start here: A data warehouse is not a database. Although you could argue that they’re both relational data systems, they serve different purposes. Data warehousing allows you to pull data together from a number of different sources for analysis and reportiong. Data warehouses store vast amounts of historical data for complex queries across all data types being pulled together.

There are lots of use cases for data warehouses. For example, if you’re doing very large amounts of data mining, a data warehouse is way better than a traditional database. Similarly, if you need to quantify vast amounts of market data in-depth, a data warehouse could help. You can use that data to understand the behavior of users in an online business to help make better decisions around services and products.

A data lake is a newer data processing technology which focuses on structured, semi-structured, unstructured, and raw data points for analysis. Data warehouses, on the other hand, only look at both structured and processes data. Where data warehousing can be used by business professionals, a data lake is more commonly used by data scientists. Data lake examples include Amazon S3, Google Cloud Platform Cloud Storage Data Lake, as well as the Azure Blog Storage service.

Data lakes are centralized storage and data repositories that allow you to work with a variety of different types of data. The cool thing here is that you don’t need to structure the data and it can be imported “as-is.” This allows you to work with raw data and run analytics, data visualization, big data processing, machine learning tools, AI, and much more. This level of data agility can actually give you some pretty cool competitive advantages.

A recent Aberdeen survey found that a well-conceived data lake lays the groundwork for an elevated level of analytical activity. For leading companies, those analytical activities also translate to substantial ROI (return on investment) in the form of business growth and higher profits. This means being able to process vast amounts of data from business-critical sources like logs, data from clicks and click-streams, social media, and even IoT data that’s funneled into the data lake.