How to secure data lakes: What you don’t know can hurt you
An enterprise data lake is a great option for warehousing data from different sources for analytics or other purposes but securing data lakes can be a big challenge.
Unlike purpose-built data stores and database management systems, in a data lake you dump data in its original format, often on the premise that you’ll eventually use it somehow. The data typically is unmanaged and available to anyone across the enterprise.
Some organizations are using data lakes as an archive for their data, some as a landing zone for data from different sources, and some as a sandbox for data scientists to play in, said Gartner analyst Merv Adrian. A lot of these lakes are moving to the cloud because of the cheap storage available from vendors such as Amazon, Microsoft, Google, and Alibaba, he said.
You put raw data into a lake and then later do some indexing, aggregation, and analysis to see what value you can extract from it.
You cannot afford to overlook the security issues that arise when warehousing all this data, especially from a regulatory compliance standpoint. Experts dive into the best approaches for securing your data lake.
To secure a data lake, you need to have a holistic understanding of the data usage, planned applications, governance requirements across those applications, and specific levels of security and access control stemming from those requirements, said Doug Henschen, principal analyst at Constellation Research.
For starters, your data lake might not necessarily even be a single Hadoop or Spark cluster anymore. Companies are maintaining and in some cases adding separate data lakes for different purposes, he said. Some, for instance, have separate lakes for ongoing production workloads and others for data science.
The manner in which organizations are deploying data lakes is also changing. Low-cost object storage options such as Amazon S3 and Microsoft’s Azure Object Store are pushing many organizations to deploy their data lakes in the cloud. For enterprises, these services are an attractive, low-cost option for archival data they might want to store as part of a data lake.
“With this growing array of options, the challenge in data lake management is ensuring not just comprehensive management across all these stores, but also authorized access and data governance,” Henschen said.
Organizations need to keep in mind that the comprehensive management and security required is often beyond the capabilities of a single platform.
Data lineage is one example. When considering data lake protections, it is important to know where data originated, how it was altered or enriched, and who touched it throughout the lifecycle of its use in decision making.
When you throw data into a data lake, realize that you don’t have many of the protections available with an enterprise database or relational database management system, Gartner‘s Adriansaid.
With traditional database management systems, the information security team might handle all the network security and access control protections but do little with the data once it enters the database management system. Data lake structures do not come with all of the governance capabilities and policies associated with a traditional database management system, from basic referential integrity to role-based access and separation of duties, Adriansaid.
One way to approach data lake security is to think of it more as a sort of a pipeline with upstream, midstream, and downstream components, said Adrian. The threat vectors associated with each stage are somewhat different and need to be addressed differently.
The upstream component is the point where you are ingesting the data going into the data lake. Go as far upstream as you possibly can and consider what threat remediation measures you can apply at those points. Consider basic things such as data classification, who owns the data, and who cares about it.
