How a Data Catalog Connects the What and the Why

“Technology is nothing in itself. What’s important is that you have faith in people, that they’re basically good and smart, and if you give them tools, they’ll do wonderful things with them.”

The what and the why 

Businesses today increasingly turn to big data and advanced analytics for their most troubling questions. The belief that if you get lots of data, a system to handle it (hardware, software), and a few smart people, the data will magically reveal insights.

But having more data doesn’t usually mean you have better data. A lot of bad data (unreliable reporting, missing observations) means a lot of noise which has to be filtered down. Second, having more data doesn’t help you determine causality between variables of interest. 

Data can tell you what is happening, but it will rarely tell you why. To effectively bring together the what and the why — a problem and its cause, in order to find a probable solution — leaders need to combine the advanced capabilities of big data and analytics with tried-and-true qualitative approaches such as human collaboration, interviewing teams, conducting focus groups, and in-depth observation. 

What can serve as a platform to do all this human collaboration? A data catalog.

A data catalog is a centralized repository of metadata, created to organize and unify data that was siloed in databases throughout the company – ERP, e-commerce stores, HR and Finance databases, and many others.

Further, it enables users to search this data using natural language. One can also discover new datasets by identifying relationship amongst data. With a data catalog users across an organization can find and understand data in a secure and governed environment. 

With use a data catalog keeps evolving both through the participants and AI.  The different roles of data catalog participants are — data organizers (data architects and database engineers), data curators (data stewards and data governors) and the data consumers (data and business analysts). 

See here for step-by-step guide to build a data catalog.

How can it help us? 

1) Enables analysts to find data from everywhere in the company – 

A data catalog collects and showcases all streams of data – qualitative as well as quantitative which can be critical for an analysis. Let’s consider an example – your company wants to understand how a loyalty program was received by customers.