Why is it so hard to change, especially from a business perspective? Why is it so hard to understand that new technology capabilities can enable new technology approaches? And nowhere is that more evident than in how companies are trying (and failing) to apply the old school “Big Bang” approach to Big Data Analytic projects.
The “Big Bang” approach likely comes from how businesses have historically gained competitive advantage – through “Economies of scale.” Economies of scale refers to the concept that costs per unit of a product decline as there is an increase in total output of said product (see Figure 1).
Historically, large organizations have leveraged economies of scale to lock smaller players and new competitors out of markets (yes, we’re talking economics again…).
The economies of scale concept morphed into an economic concept called the Learning Curvewhich was employed by companies such as Texas Instruments and DuPont in the 1970’s. Like economies of scale, a learning curve exploits the relationship between cost and output over a defined period of time; that is, the more someone or an Organization learns from the repetitive completion of a task, the more effective they become in the execution of that task.
Thepopular book “The Lean Startup” by Eric Ries expands upon the Learning Curve concept introducing the concept of incremental learning to become more efficient more quickly. “The Learn Startup” shares a story about stuffing 100 envelopes. From the story, we learn that the optimal way to stuff newsletters into envelopes turns out to be one at a time, versus folding all the newsletters first, and then stuffing all the newsletters into the envelopes, and then sealing all the envelopes and finally stamping all of them. The reason the “one at a time” or incremental learning approach is more effective is because:
From an economics perspective, we have to ask a very hard question:
Is “incremental learning” (and the immediate application of that incremental learning) more important than “economies of scale”?
“Big Bang” Big Data Analytics projects are poisonous (in my humble opinion) to the success of your Big Data and Digital Transformation journey. Organizations do not have to spend $20M+ on a 3-year Big Data Analytics project and then hope that they get what they thought they bought at the end of the project.
