Starburst adds features to further data mesh approach
Data mesh specialist Starburst on Tuesday launched three new data discoverability and governance features for its Galaxy platform. It also revealed that its Enterprise platform now supports AWS Lake Formation.
Starburst, founded in 2017 and based in Boston, unveiled the new capabilities at AWS re:Invent 2022, a user conference hosted by cloud computing giant Amazon Web Services. The conference was held in-person in Las Vegas with some content also available to remote attendees.
Starburst offers Galaxy for its SaaS analytics customers and Enterprise for its on-premises users. Both are designed to let customers build a data mesh architecture, which is a decentralized approach to data management and analytics.
The approach enables data teams within different domains — for example, marketing or human resources — to manage and analyze their own data rather than have a centralized data team do all the data management and analysis for their entire organization.
Data mesh uses the domain knowledge of an organization’s workers, with the theory being that an expert in marketing or human resources data will be better at working with their domain’s data than a data generalist. Its intent is reducing the demands placed on a centralized data team that often result in bottlenecks.
But rather than isolate an organization’s data within each domain responsible for its own data, data mesh connects an organization’s data with data catalogs and other data integration tools that enable sharing and collaboration.
Like Starburst, Talend and Informatica are among the vendors offering data mesh tools.
The three data discoverability and governance capabilities now part of Galaxy are aimed at helping organizations develop analytics tools, such as charts and dashboards, that let users analyze their data and reach insights that lead to action, according to Starburst. Before the new features, Starburst users could see their data. But they were given no insight into what data was valuable and what to do with that data. Instead, they had to connect their data to other products such as Collibra and Alation — both data catalogs — to gain perspective. With new data discovery capabilities, Starburst now enables analytics users to better search and understand their data by automatically populating the data with metadata that gives greater context, including how it is being used by others. To simplify the extract, load and transform process for data engineers, new automated schema discovery features enable users to discover new datasets whether they’ve been transformed or not. Finally, granular access control capabilities allow data administrators to clearly see and understand which users can access what data and both monitor and change permissions as needed. Combined, the capabilities are a significant improvement for Starburst users, according to Doug Henschen, an analyst at Constellation Research. “This is a notable step up in metadata management capabilities for the Starburst Galaxy service,” he said. “This brings core discovery, schema and access-control capabilities right into the product and may satisfy the needs of a majority of customers.” Customers in highly regulated industries, however, may still require the more advanced data governance capabilities provided by specialists like Alation and Collibra, Henschen added. Meanwhile, of the three new capabilities, Henschen noted that data discovery could prove most useful, calling it “the most fundamental to the broadest base of potential users.” Russell Christopher, Starburst’s director of product strategy, highlighted the new data discovery capabilities. “The discoverability features — for me, as an administrator at times and a consumer at times — allow me to go really fast from a very high level down to a very deep granular level of the data,” he said. Christopher noted an important aspect of the new discoverability and governance features is that they reduce the need for third-party tools.
