5 Reasons to Love Snowflake’s Architecture for Your Data Warehouse

You may ask, “So, how does Snowflake do data warehousing differently than any other solution available? Glad you asked. The answer is, architecture.

This blog details five characteristics about our multi-cluster, shared data architecture that are reason enough to move your on-premises data warehouse and analytic workloads to the cloud. If you already have a data warehouse in the cloud, but it’s a “cloud-washed” data warehouse, the imperative is the same.

For the last decade, you’ve likely heard from various technology companies that the best and most cost-effective way to quickly create insights from a mix of data, especially a mix of multi-structured data (JSON, Avro, tables, etc.), is to move the analytics to the data. Figure 1 represents the typical MPP or cluster architecture. The theory goes, if you tie data and compute resources together and spread them across commodity hardware through massively parallel processing (MPP), or perhaps Hadoop on distributed servers, analytics would be swift and low cost.

However, the passage of time and well publicized user frustrations have proven that tying compute to storage creates a number of negative business consequences. Tuning, for one, becomes a chore. Scaling the environment is disruptive, query concurrency is limited and  supporting separate workloads simultaneously (e.g., new data loading and BI queries) is nearly impossible. In addition, accessing data in order to query it frequently requires engaging your organization’s data engineering professionals with the Java, Python or Scala language skills necessary to extract the data from the clusters.

But going through the data engineering team often means delays. It’s not the sort of process to have in place if you want to be a nimble, fast-moving, data-driven organization. Further, executives would much rather have data engineers focused on developing new products and services.

Now, let’s dive into Snowflake’s multi-cluster, shared data architecture and the reasons why you’ll love it.

Snowflake was founded on the belief that tying compute and storage together is not an effective approach for limitless, seamless scaling. Snowflake’s multi-cluster, shared data architecture (See Figure 2), separates compute resource scaling from storage resources, thus enabling seamless, non-disruptive scaling.

In Snowflake, while queries are running, compute resources can scale without disruption or downtime, and without the need to redistribute/rebalance data (storage). Scaling of compute resources can occur automatically, with auto-sensing. This means the Snowflake software can automatically detect when scaling is needed and scale your environment without admin or user involvement.

For storage, you can scale resources to virtually any capacity without the extra cost of adding unnecessary compute resources. Both resource groups can scale up or down – including suspend in the case of compute resources. With Snowflake, you can easily and seamlessly customize resources to the specific needs of your organization.

Traditional MPP clusters in the cloud may require you to place your database into read-only mode or may require you to take the cluster offline. With other architectures, if you are able to scale compute “automatically”, you still have to manually engage the system to tell it to scale. This is not seamless.

Other cloud data warehouse offerings are beginning to separate compute and storage. But, from a cloud data warehouse perspective, separating compute and storage layers is not enough. If all that was accomplished was the separation of compute and storage, you’d have a topology that resembles the server plus storage networking (either LAN or SAN) topologies typical of on-premises data centers that, in some cases, date back to the 1990s.