Business Applications, Data Integration, Master Data Management, Data Warehousing, Big Data, Data Lakes, and Machine Learning; these all have (or should have) a common and essential ingredient:Â A Data Model; Let us NOT forget about that; Or, as in many situations I run into, ignore it completely!
The Data Model is the backbone of almost all of our high value, mission critical, business solutions from e-Commerce and Point-of-Sale, through Financial, Product, and Customer Management, to Business Intelligence and IoT. Without a proper Data Model, where is the business data? Probably: Lost!
After the success of my Blog Series on Talend Job Design Patterns & Best Practices (please read Part 1, Part 2, Part 3, and Part 4), which covers 32 Best Practices and discusses the best way to build your jobs in Talend, I hinted that data modeling would be forthcoming. Well, here it is!
Data Models and data modeling Methodologies have been around since the beginning of time. Well, since the beginning of computing anyway. Data needs structure in order to make sense of it and provide a way for computers to deal with its bits and bytes. Sure, today we deal with unstructured and semi-structured data too, but for me it simply means that we evolved to more sophisticated paradigms than our computing predecessors had to deal with. The Data Model therefore remains, and provides the basis upon which we build highly advanced business applications. Like the Talend best practices, I believe we must take our data models and modeling methods seriously.
Looking back at the history of Data Modeling may enlighten us, so I did some research to refresh myself.Â
In the ‘Computing Dark Ages’, we used flat record layouts, or arrays; all data saved to tape or large disk drives for subsequent retrieval. However, in 1958, J. W. Young and H. K.
