Battling Data Demons with Data Governance

The stories you hear on the news often mention how this person or that person was battling their demons. Alcoholism, drug addiction, compulsive behaviors like gambling … Demons take many forms. They almost never result in good things happening. The word “demon” does not describe something that inspires you to do something good – and that should be what we hear and read about. But demons rule the day.

Organizations have demons too. Just like people, some organizations are battling those demons and some know they have demons and do not attempt to address them. Organizational demons may be politically-based, culturally-based, gender-based or even … Data-based.

Data demons ‒ defined (by me) as data-related behaviors that we know are wrong and we continue to employ ‒ run amuck in many organizations. Many organizations work on these demons daily and many have a difficult time acknowledging that they have demons. Let’s look at the forms these data demons take, and what can be done to address them.

Let’s start where data is defined. Data definition takes place throughout data modeling exercises, software package implementation, application development, incorporation of external big data sources, and by going back and defining databases and system or data warehouse data that has evolved (read “become bastardized”) and been used over time.

Data should be well-defined at the beginning of its lifecycle so that it is well understood throughout that lifecycle. Sufficient data definition requires thorough collection and development of data requirements, which is one area where organizations have data demons. Taking the time to collect business data requirements and develop sound business definitions for data has not always been the focus of traditional information system development methods. Agile development efforts have further complicated matters by requiring quick and incremental delivery of complex information systems, leaving little time to flush out data problems while accumulating data debt.

Data Governance can battle Data Definition Demons in many ways. Data Governance calls for “executing and enforcing authority over data,” meaning that authority must be used to make certain that data is defined to the degree that it will assist the organization to get the most out of that data.

Data Governance can battle Data Definition Demons by formalizing the involvement of the “right” people at the “right” time in the data definition process and by getting the “right” people to authorize that a data definition is thorough and complete.

Data Definition includes many types of metadata including business description and naming of the data, data lineage and location, business rules, and compliance and handling rules … Whatever the organization determines is important to squeeze maximum value out of their data.

Data Definition Demons become evident when the data definition is not complete or shareable / shared with the business users of that data. If business people do not understand the data, know where they can find the “right” data, and don’t know how they can use the data, Data Definition Demons are among you.

Data must be produced to meet business needs. The production of data can have demons too. Data Production Demons rear their ugly heads when the people or process responsible for producing data do not understand why the data is being collected and how it will be used. Cashiers entering the store’s zip code rather than the customer’s zip code, office staff reordering patient diagnosis codes, accepting default data rather than entering correct data and sharing data that is supposed to be held private are all results of people not being held formally accountable for how they are producing data.