Busting 5 Myths about Data Lakes
- by 7wData
The data lake is still quite new, so it's natural that a few myths and misunderstandings have proliferated across the data management community. To set the record straight, I'd like to bust five myths that I keep hearing. First, I need to define the data lake so that we're all on the same page. A data lake is a user-defined method for organizing large volumes of highly diverse data. The data of a lake may be deployed on diverse data management platforms, including Hadoop clusters, relational databases, clouds, or a combination of these. Depending on the platform, a data lake may handle diverse data types, ranging from unstructured to semistructured to structured data. For most organizations, a data lake supports multiple use cases, including broad data exploration, advanced analytics, data warehouse extensions, and data landing and staging. Lakes may also serve specific departments (marketing, supply chain) or industries (healthcare, logistics). Busted! Although it's true that any database can become a dumping ground, this isn't what successful early adopters are doing with their data lakes. Lake owners interviewed by TDWI say that a data lake is a balancing act. Some users are allowed to dump, but others are not. For example, data analysts, data scientists, and some power users need to create "sandboxes" of data in their work; they are allowed to bring data into and out of the lake freely as long as they govern themselves. Most other users must petition the lake steward or curator, who vets incoming data. Myth #2: Data lakes are only for Internet firms. Busted! Hadoop and the data lake were pioneered by Internet firms, and we owe them our thanks for those innovations. However, TDWI has found data lakes in production in several mainstream industries, including finance, insurance, telco, pharma, and healthcare. As noted above, some lakes serve departmental operations or analytics. TDWI has also found multiple forms of analytics operating on a lake data, including data and text mining, clustering, graph, and predictive analytics, and natural language processing. Lake-based analytics supports a number of analytics application types, including risk calculations, customer segmentation, and the detection of fraud, security breaches, and insider trading. Data lakes serve a wide range of enterprises, and each lake is typically multitenant because it serves multiple business units and use cases. Busted! A recent TDWI survey showed that over half of data lakes in production are on Hadoop exclusively (53 percent).
[Social9_Share class=”s9-widget-wrapper”]
Upcoming Events
Shift Difficult Problems Left with Graph Analysis on Streaming Data
29 April 2024
12 PM ET – 1 PM ET
Read MoreCategories
You Might Be Interested In
The Hospitality Industry Benefits From the Emergence of Big Data
1 Aug, 2017When it comes to adopting technology as well as evolution driven by data, the hospitality industry is known to lag …
How to choose a data science vendor
13 Jan, 2018Data science, big data and AI are current buzzwords, and many companies are rebranding their business intelligence products with these …
What Lawyers Want Everyone to Know About AI Liability
26 Feb, 2021There are more discussions about AI ethics and responsible AI these days, but companies need to be clear about potential …
Recent Jobs
Do You Want to Share Your Story?
Bring your insights on Data, Visualization, Innovation or Business Agility to our community. Let them learn from your experience.