Cloudera and Intel have a long history of innovation, driving big data analytics and machine learning into the enterprise with unparalleled performance and security. We are pleased to build upon that direction with our collaboration on Intel® Optane™ DC persistent memory. Available to customers running 2nd Generation Intel® Xeon® Scalable processors, Intel Optane DC persistent memory can significantly enhance the performance of real-time and streaming applications. Data-driven organizations – particularly those utilizing IoT – require low-latency, high-performance compute to process data at the edge in order to make faster, smarter decisions.
Intel Optane DC persistent memory is a new tier in the memory and storage hierarchy located between DRAM and Solid State Drives with latency closer to DRAM (see Figure 1). It uniquely combines high capacity, low latency, and non-volatility into a single package and is offered in 128GB, 256GB, and 512GB modules, expected to be at a lower cost-per-gigabyte and larger memory capacity compared to large DRAM DIMMs.
Intel Optane DC persistent memory was designed with ease of adoption in mind and therefore can be configured in two differentoperating modes: 1. memory mode, and 2. app direct mode. In memory mode, done with a simple BIOS change, the operating system simply sees the memory as a large pool of volatile memory similar to DRAM. When a data request is made the memory controller will first check for the data in DRAM, if not present, then it checks the Intel Optane DC persistent memory, with only slightly longer latencies. When in memory mode, the data is not saved in the event of a power loss. In app direct mode, the application can utilize DRAM for operations that require low latency without persistency and leave Intel Optane DC persistent memory to handle large data structures at memory bus speeds without volatility. It is possible to enable a portion of persistent memory for app direct mode while the other runs in memory mode.
Cloudera customers who want more flexibility in how and where they run their applications can benefit from Intel Optane DC persistent memory as well. A key characteristic of an enterprise data cloud is its ability to run multiple workloads on shared data without encountering “noisy neighbor” problems. This is achieved through an architecture that fundamentally separates compute from storage. Intel Optane DC persistent memory mitigates the I/O bottlenecks associated with these bifurcated environments.
Apache HBase® is one of many analytics applications that benefit from the capabilities of Intel Optane DC persistent memory. HBase is a distributed, scalable NoSQL database that enterprises use to power applications that need random, real time read/write access to semi-structured data.
