Hortonworks enters joint initiative with Hewlett Packard Enterprise on Apache Spark enhancements

Hortonworks and Hewlett Packard Enterprise, aka HPE, announced a joint initiative to produce new enhancements to Apache Spark, specifically to handle a new class of analytic workloads from large pools of shared memory.

So far, the collaboration has led to enhanced shuffle engines for faster sorting and in-memory computations, and improved memory utilization.

“We’re hoping to enable the Spark community to derive insight more rapidly from much larger data sets without having to change a single line of code,” said Martin Fink, EVP and CTO, Hewlett Packard Enterprise, and Hortonworks board member, in the announcement.

“We’re very pleased to be able to work with Hortonworks to broaden the range of challenges that Spark can address.”

The two companies plan to contribute the resulting tech to the Apache Spark community.

Hortonworks also announced enhancements to its Connected Data Platforms.;