Data Science vs Business Intelligence: same but completely different

Business Intelligence is an umbrella term that describes concepts and methods to improve business decision making by using fact-based support systems [1]. Modern Business intelligence is not just business reporting. It is a mature system that provides interactive dashboards, what-if planning, mobile analytics, etc. It also includes large back-end parts for maintaining control and governance around reporting.

Since BI is an umbrella term, it can be different from company to company. For some, it can be just a basic Key Performance Indicator (KPI) reporting with all supporting infrastructure, other companies may use advanced predicting methods based on statistical models and advanced tools. But regardless of methods or tools used — they provide facts for decision making to business stakeholders, according to their requirements.

From a Business Process standpoint, there is not much difference between Data Science and Business Intelligence — they both support business decision making based on data facts. Probably this is why it is often assumed in businesses starting their first Data Science or AI projects, that Data Science is the same old Business Intelligence that works much more cleverly. From this assumption, it follows that a Data Science project can be done on the top of existing BI infrastructure and processes.

This is when problems begin: it turns out that a Data Scientist wants data out of the system as a CSV file, but company security policy will not allow that; Data Scientists are building their models using weird libraries and software which infrastructure teams do not and will not support in production, and so on.

Any first Data Science project in any company will start generating new challenging requirements to existing teams right from the start. How come? Why this is happening if Data Science does the same as BI does? Why those weird requirements?

The difference is in the type of questions that they address: BI provides new values of previously known things, using some formula that is available. Data Science works with the unknown (see the first part of this series), answering data questions that nobody have answered before and, therefore, without formula in hand.

In BI, business comes to a BI developer with a formula or a method of calculating a report or a KPI, that business owns. This means that business has designed the BI method and that they understand and are comfortable using it.

In Data Science it is quite different: business comes with their actual data and some question that has never been answered before. It is now up to a Data Scientist to test multiple approaches and select the best one, balancing between accuracy, simplicity, usability, and capabilities of a production platform. Once the model is selected and agreed with the business, it becomes a known method for answering the question, it becomes a subject of Data Analytics rather than Data Science.

You may notice that above statement about BI is debatable — it does not deal with completely known things — it may have a formula or a method, but it calculates unknown KPI values or even makes predictions using approved methodology. To explain this duality, I’m using a nice concept of known unknowns and unknown unknowns, that was popularised by US Secretary of Defence Donald Rumsfeld back in 2002 in his famous answer about lack of evidence linking the government of Iraq with the supply of WMD to terrorists.

Using this concept, I could now formulate the difference much shorter:

This is the biggest and fundamental difference between them. At first, it may seem a pure formalism, focusing on a difference that is not that significant, but it will change once you start thinking about the consequences.

Firstly, dealing with unknown unknowns, Data Science cannot guarantee success in the very beginning of a project, predict what the solution would look like and how difficult it will be to implement.