Introducing Cloudera DataFlow (CDF)

Late last year, the news of themerger between Hortonworks and Clouderashook the industry and gave birth to the new Cloudera – the combined company with a focus on being an Enterprise Data Cloud leader and a product offering that spansfrom edge to AI. One of the most promising technology areas in this merger that already had a high growth potential and is poised for even more growth is the Data-in-Motion platform calledHortonworks DataFlow (HDF). It is a key capability that will address the needs of our combined customer base in areas of real-time streaming architectures and Internet-of-Things (IoT). HDF is already a highly successful product offering with hundreds of customers likeClearSense,Trimble,Hiltonetc.

So, what happens to HDF in the new Cloudera? What should customers expect? The good news is that it will still remain strategic to our company as well as to our customers. So, HDF is now reborn as Cloudera DataFlow (CDF).

Cloudera DataFlow (CDF) is a scalable, real-time streaming data platform that collects, curates, and analyzes data so customers gain key insights for immediate actionable intelligence. It meets the challenges faced with data-in-motion, such as real-time stream processing, data provenance, and data ingestion from IoT devices and other streaming sources. Built on 100% open source technology, CDF helps you deliver a better customer experience, boost your operational efficiency and stay ahead of the competition across all your strategic digital initiatives.

With the rise of streaming architectures and digital transformation initiatives everywhere, enterprises are struggling to find comprehensive tools for data management to handle high volumes of high-velocity streaming data. CDF, as an end-to-end streaming data platform, emerges as a clear solution for managing data from the edge all the way to the enterprise. It can handle edge data collection, data ingestion, transformation, curation, data enrichment, content routing, processing multiple streams at IoT scale and analyzing those in real-time to gain actionable intelligence. CDF can do this within a common framework that offers unified security, governance and management.

The key aspects of the CDF platform are –

Edge Data Management – Set up hundreds of MiNiFi agents in or near edge devices to enable edge data collection, content filtering, routing etc. This allows you to take on complex, distributed use cases such as connecting hundreds of retail stores across the country or getting data from thousands of utility sensors from your consumer edge. This is going to be a significant area of investment for us given our customer interest, the industry trends and the market potential. Flow Management – Adopt a no-code approach to create visual flows for building complex data ingestion / transformation with drag-and-drop ease. Powered by Apache NiFi and its 260+ pre-built processors, CDF enables you to take on extremely high-scale, high-volume and high-speed data ingestion use cases with simplicity and ease.