With DataPlane, Hortonworks aims to help companies drowning in data lakes
When the term “data lake” was coined in 2011, the notion was that organizations needed a single pool for all their data, so it could be tapped for whatever analysis or other application it was needed for, instead of languishing in countless data siloes. But now, it’s common for enterprises to have multiple data lakes across their own data centers and multiple cloud computing providers, partly defeating the purpose of the whole idea.
That’s the problem big-data company Hortonworks Inc. aims to help solve with a new service it’s announcing today. The Hortonworks DataPlane Service, or DPS, is a cloud offering that aims to corral data in multiple data lakes and other data repositories, whether it came from customer transactions in the U.S., a security cam in Japan or anywhere else.
The company said the service, built upon open-source technologies such asApache Atlasfor managing the security and usability of data, will make it simpler for companies to set up and operate distributed data systems, whether they’re used for data science, analytics or data warehousing — both so-calleddata in motion and data at rest. That’s becoming especially critical as data from far-flung Internet of Things devices explodes and companies need to do data-intensive work on it such as artificial intelligence and machine learning for applications ranging from voice and image recognition to self-driving cars.
“It’s clear that data is not going to be in one place,” Hortonworks co-founder and Chief Product Officer Arun Murthy (pictured) said in an interview. “This is the first time that anyone has acknowledged that there will be multiple data lakes.”
The DataPlane service, he said, is a way to stitch together all the data in various data lakes and beyond. With DataPlane, companies get consistent data governance across data lakes. That’s important, Sesh Rangarajan, senior director and head of analytics at Liberty Mutual Benefits, said in a statement: “We require a single data fabric to make sense of it all.”
Indeed, as Noel Yuhanna, an analyst at Forrester Research Inc., described it to SiliconANGLE, it’s turning his notion of a “data fabric” that weaves together many data threads into a reality.
