The dos and don’ts of data lakes

“In 3 keys to keep your data lake from becoming a data swamp” Thor Olavsrud provides CDOs, CIOs, and other business intelligence leaders with relevant guidance on preventing the so called ‘data swamp’. This includes:

These are great suggestions — in particular, having a business mission for your data lake.

Clearly, delivering a data lake rather than a data swamp is important because insight-driven companies make more money and develop more sustainable barriers to entry. Forrester Research estimates that “Insights-driven businesses will steal $1.2 trillion a year by 2020” (“Insights-Driven Business”, Forrester Research, July 27, 2016).

The insight-driven accomplish this by experimenting and continuously learning. These firms are adding data lakes often at the same time as they put in place CDOs. They understand in particular that advanced data capabilities are needed to implement a successful digital transformation”(“Data Centric Businesses need a Data Centric Leader”, Forrester Research, April 26, 2016). Successful CDOs clearly need to lean on their CIOs to succeed at creating a valued data lake.

To further understand how to avoid a data swamp, I interviewed 12 leading edge CIOs; specifically, I asked them for recommendations regarding the dos and don’ts of data lakes.

CIOs see real opportunity to create new business value from data lakes and self-service business intelligence. They believe these trends matter because they are about increasing the availability and transparency of data and enabling business users with the ability to get answers without involving their IT colleagues. Most CIOs connect self-service business intelligence to the data lake, which means effective data lakes are not just about data storage, but also about citizen and professional data scientists’ self-exploration of the information contained within them.

Some CIOs even see self-service business intelligence options broadening the community served by IT:

“Self Service BIallows our broader community to be engaged in relevant and up-to-date analytics-based decision making”, CIO at Binghamton University

While some see the valuable impact of taking this step when data lakes are used for every day decisions, Joanna Young, former CIO of Michigan State University suggests that CIOs shouldn’t “use new tools to pave old reporting cow paths.” Several CIOs, however, suggest self-service business intelligence and data lakes offer better business and IT alignment all by themselves.

Taking these comments on board, I would like to suggest five things that will keep you out of the murk of a data swamp:

CIOs suggest that they have learned from the first wave of business intelligence and value is generated only when asking the right questions are asked. They are candid that even though the tools make data highly available, asking the right questions is still a challenging process for most organizations. Some worry that if data lakes don’t move from the experimentation phase to generating business value, that CEOs and CFOs will start complaining and heads may roll. For this reason, it came as no surprise when one CIO said that a data lake with no business goals or purposes is just taking up space.

CIOs say even though it is not easy that IT shouldn’t always say yes to a data lake project. This sentiment was also found in The Big Data Payoff, Capgemini IDG. 2016 where interviews with 210 business executives showed that those who excel with big data use it to achieve strategic business objectives

This echoes Tom Davenport who said, “Even the most analytically oriented company needs to target its analytical efforts where they will do the most good, because resources, especially talent, are always constrained.”

CIOs need to help make sure their business customers start with an end in mind and be clear that a “data first, questions later” approach won’t work.