Are you comfortable with source systems feeding ETL processes into operational data stores or master reference data through an enterprise service bus with the product, supply chain and business operational reports dumped into a presentation layer with soft analytics, dashboards, alerts and scorecards? That was yesterday.
Don’t get caught, explaining your new data warehouse initiative with old terminology.
Data warehouses are not designed for transaction processing. Modern data warehouses are structured for analysis. In data architecture Version 1.0, a traditional transactional database was funneled into a database that was provided to sales. In data architecture Version 1.1, a second analytical database was added before data went to sales, with massively parallel processing and a shared-nothing architecture. The challenge was that this resulted in slow writes and fast reads. In data architecture Version 2.0, the transactional database populated a second database which flowed into a third analytical database, which connected to the presentation layer (business intelligence). In data architecture Version 2.1, multiple transactional databases fed the core database which provided information downstream to data stores (sales, marketing, finance) that connected to a business intelligence engine. At this point, traditional database structures end and modern structures begin: data architecture Version 3.0.
The two below examples highlight the difference between a traditional Data warehouse and a data a modern Data warehouse (using Hadoop for this example).
Most database designs cover four functions: 1) data sources, 2) infrastructure, 3) applications and 4) analytics. This principle of design does apply to both traditional data warehouses and modern architectures. The design thinking, however, is different. In a modern data warehouse, there are four core functions: 1) object storage, 2) table storage, 3) computation and processing, and 4) programming languages.
The lack of data governance, inadequately trained staff, weak security and non-existent business cases each factor into why data warehouse or business intelligence initiatives fail to achieve the desired outcomes. Keep your data warehouse program on track.
Start by strengthening your framework for business intelligence.
