If Data is as Valuable as Gold, It’s Time to Polish Your Data Architecture

It speaks volumes of the world we live in today when headlines such as “The world’s most valuable resource is no longer oil, but data” and “Why Data May Be More Valuable Than Dollars” are commonplace. With the explosion of IoT and with that 2.5 quintillion bytes of data being created per day, the underlying power of this data comes as no surprise.

Unlike gold however, data is ubiquitous and being created at an exponential rate. So where’s the value in something that is everywhere? Unlike its “comparative commodity” like gold or money, the value does not come from the volume of data you have (although this is a good starting point!), rather the value comes from the insight you can gain from the data that you already have.

So how do you go about gaining insight from your data? This is the important question and the final goal that you need to keep in mind.

Previous methods of gaining insight from data include ETL (Extract, Transform, Load) which involved copying the data and loading it into a data warehouse. Data could then be extracted from this “data archive” to gain a business advantage.

While this method has been successful in the past, the need for the faster delivery of data insights is overtaking the capabilities of ETL and data warehousing for the following reasons:

The ETL process involves making copies of the data and then physically moving this copy and loading it into the data warehouse. This process, as well as the costs associated with storing the data in the warehouse, are very expensive.

Making copies of the data is problematic because you end up with a number of versions scattered across the enterprise which can quickly become unmanageable. Furthermore, if various teams are working independently from each other making changes to these copies, it becomes nigh on impossible to differentiate the original from the latest versions.