There’s a new record holder in the world of “big data.”
in-memory computing is comprised of two main categories: in-memory Databases and In-Memory Data Grids. I’d like to delve into the differences between the two groups.
Nikita Ivanov, CTO of GridGain
- Nomenclature
- Tiered Storage
- Technical Differences
- Speed Only Vs. Speed + Scalability
- Replace database Vs. Change Application
- In-Memory Data Grid
- In-Memory database
Tiered Storage
The main point to take away is that “in-memory” products are not confined to one fixed definition, but in the end they all have a significant “in-memory” component.
Technical Differences
The majority of IMDBs are RDBMS that store data “in memory” instead of disk. With only a modest list of unsupported SQL features, IMDBs generally provide good SQL support. IMDBs are also shipped with ODBC/JDBC drivers and can be used in place of existing RDBMS often without significant changes.
Conversely, In-Memory Data Grids typically lack full ANSI SQL support; but instead provide MPP-based (Massively Parallel Processing) capabilities where data is processed in parallel fashion and spread across a large cluster of commodity servers. The main access pattern is key/value access, MapReduce, several forms of HPC-like processing, and restricted distributed SQL querying and indexing capabilities.
Speed Only vs. Speed + Scalability
One of the key differences between IMDGs and IMDBs is in the level of scalability allotted. Due to IMDG’s MPP (Massively Parallel Processing) architecture, it has an inherent capability to scale to hundreds and thousands of servers; on the other hand, due to the fact that SQL joins cannot be efficiently performed in a distribution context, IMDBs are explicitly unable to scale horizontally.
This is the dilemma with in-memory databases – SQL joins, one of IMDB’s most useful features, is also what limits scalability. This accounts for why most existing SQL databases (disk or memory based) are based on vertically scalable Symmetrical Processing architecture, unlike IMDGs that utilize the much more horizontally scalable MPP approach.
Replace Database vs. Change Application
In addition to scalability, another key differentiation between IMDGs and IMDBs involves switching out existing databases versus altering the application.
IMDGs always work with an existing database, providing a layer of massively distributed in-memory storage and processing between the database and the application, which can then be relied on for rapid data access and processing. Most IMDGs are highly integrated with existing databases, and can seamlessly read-through and write-through to and from databases when necessary.
