A new era of SQL-development, fueled by a modern data warehouse
SQL development is not a new concept. However, as the data warehousing world shifts into a fast-paced, digital, and agile era, the demands to quickly generate reports and help guide data-driven decisions are constantly increasing. This puts new pressures on the people working behind the scenes to prepare and serve data in a consumable way to a growing audience with various levels of access credentials and technical expertise. It also puts pressure on tooling and technology platforms to enable self-serve BI in an easy, yet secure and controlled way.
Consider the following:
These trends and demands lead to stress for existing data warehouse solutions – scale, efficiency, security integrations, IT budgets, ease of access. The stress is also reflected by end users and SQL developers on how efficiently they are expected to serve the business.
Cloudera recently launched Cloudera Data Warehouse, a modern data warehousing solution. It is designed and optimized to help with the majority of stress elements outlined above. It also comes out of the box with a widely adopted, easy-to-use SQL Developer Workbench: Hue, that is specifically designed for the needs of the modern knowledge worker:
In the latest release of Cloudera Enterprise (C6) we enabled Hue 4.0 by default, which aims to expedite and simplify the common tasks for the modern SQL user. Here are some highlights:
Most data is ingested through data engineering pipelines. Cloudera has a number of fully integrated tools such as Sqoop, Flume, Kafka, cloud service options, and optimized partner solutions from Informatica and Streamsets to satisfy our customer’s needs. But for an SQL user, it is also common to have “data laying around” – some flat files on S3, some tables in an external DB. Bringing in tables or files can now easily and in a guided way be done through Hue, which connects to MySQL, S3, ADLS, and other backends to streamline the task of ingesting important additional data sets.
Finding the right data to work with can be daunting for a large organization. We have implemented ways that will expedite data discovery as well as exploration. In the simplified data browser in C6 the end user can easily search for databases, namespaces, tables, views, collections, and even files. You can also easily preview and sample the data to validate you have found the relevant assets to work with.
A unique integration with our Cloudera Navigator tool (which is part of the Cloudera Data Warehousing portfolio and helps with catalogs, lineage, and auditing) allows us to show you the most commonly used tables (crowdsourced) – very helpful information that will give an indication of which table is most likely the one to work with if you are not entirely sure. It has been estimated to shorten the discovery phase by hours.
