How Big Data Scientists Can Benefit from Human Judgment

The quality of your data determines the quality of your insights from that data. Of course, the quality of your data models and algorithms have an impact on your results as well, but in general it is garbage in, garbage out. Therefore, (Total) Data Quality Management (DQM) or Master Data Management (MDM) have been around for a very long time and it should be a vital aspect of your data governance policies.
Data governance can offer many benefits for organizations, including reduced head count, higher quality of data, better data analysis and time savings. As such, those companies that can maintain a balance of value creation and risk exposure in relation to data can create competitive advantage.
Garbage in, garbage out. Especially with the hype around artificial intelligence and machine learning, that has become more important than ever. Any organization that takes itself serious and employs data scientists to develop artificial intelligence and machine learning solutions should take the quality of data very serious. Data that is used to develop, test and train algorithms should be of high quality, high volume and should be well organised and enriched. The right high-quality data will result in better and smarter algorithms that will return better insights and results.
The problem is, however, that improving and optimising your data to develop better machine learning algorithms is difficult, time-consuming and very expensive if done by highly-talented data scientists. While it is a necessity, it does not have to cost the world and if done correctly, can actually free-up your data scientists to focus on developing better machine learning algorithms. How, you might wonder? By turning to the crowd to help you train your algorithms with the right high-quality data, that is cleaned, parsed and enriched.
A few years ago, Andrew McAfee proposed a simple rule for the second machine age: as the amount of data goes up, the importance of human judgment goes down. McAfee argues that we should turn most of our decisions over to algorithms; algorithms that have access to vast troves of data and are based on mathematical models which are not bothered by human traits such as emotions or feelings. As such, algorithms are a lot better at making informed decisions than humans. But, McAfee also argues that when expert opinions are quantified and added to the algorithm, the quality of the outcome generally goes up. As such, human judgment becomes important as an input to algorithms, but not as a replacement for algorithms.
Practically speaking, human judgments, as part of the input for algorithms, have increasingly become important for data scientists to develop good algorithms and models. Human judgment is, in fact, essential at the initial design stages of the models. Algorithms require human interpretive skills to improve the outputs of the machine learning models and human judgment can help improve the quality of the data and as such improve the outcome of the model. Although McAfee argued that the importance of human judgments goes down, in fact it has become more important than ever. Next to human judgments becoming more important, the data governance is also increasingly seen as an area for competitive advantage.
Data governance enables firms to comply with regulations and proper data governance policies can offer you a competitive advantage. Data governance comprises of several important areas: data principles, data quality, metadata, data access and data lifecycle:
Data principles define how business users can manage and deal with the data available. Data quality refers to the accuracy, timeliness, credibility and completeness of the data.


