An interview with Robert Fink, Architect of Foundry, Palantir’s open data platform Part One: Open Data Architectures

Editor’s note: This post is the first of a series of three capturing the result of recent interview/discussions I had with Robert Fink of Palantir. The conversation was wide ranging, hitting on topics of design, development environments and a bit on the philosophy of enterprise tech. Several common themes emerged in those topic areas, including ways that Palantir has been leveraging open approaches to data architectures, system design and even developer environments. This first post focuses on open data architectures.– bg

Bob Gourley: Robert I look forward to your thoughts on three related topics, open data architectures, open-source software, and open development approaches. But first could you let us know a bit about your background?

Robert Fink: I am a computer scientist at Palantir Technologies in Palo Alto, California. I have a background in Physics, and a PhD in Computer Science from Oxford University specializing in database theory. Here at Palantir I am the architect of our open data platform that we call Foundry. Today I spend most of my time on the design and architecture of Foundry. I also oversee our work with the open-source software community… as you know, software development is a team sport these days. I’m also interested in developer education and in development best practices across the industry.

BG: I’m intrigued that you call Foundry an “open data platform”. Is it a database? Or an application? Or an enterprise service bus? Or an ETL service? Or some combination of those?

RF: You managed to mention everything that Foundry is not! Or at least not in the context of the traditional interpretations of these concepts. Foundry is the culmination of 15 years of experience with data integration pipelines and data analysis workflows across our government and commercial customers. We had initially built Foundry as a productivity tool for our own engineers, but soon realized that Foundry allows enterprises to reimagine how they interact with data. As implied by your question, legacy approaches to data involve awkward kludges of components all playing some siloed role in making data more useful to the enterprise. We designed Foundry to work openly with all data sources and repositories in ways that smoothly integrate data, create and capture new knowledge, and make such knowledge available to all users of the platform, to the entire enterprise. Doing this requires an open platform that makes it easy for enterprises to move data in and out. But more than that, it requires an open platform that lets any usersource, fuse and transform data using incredibly powerful tools.

BG: So, if an enterprise has developed other tools or uses other commercial solutions, will Foundry work with those?

RF: Yes, of course. Over the last 15 years we have always had APIs that allowed data to be exported to other tools, but heard consistently from our enterprise clients that faster and more streamlined or native interoperability is required. Our current system was designed from the ground up with this open data architecture, and we migrated Gotham APIs to modern, interoperable standards. This gives customers the ability to use any tool now, and, as the world changes and new innovations become available, our solution enables enterprises to more rapidly innovate and adopt new technologies as they become available. Designing as an open system gives that benefit.

BG: So what would you say the greatest value add of Foundry is?

RF: In my experience, the biggest benefit to IT organizations lies in the data management and governance capabilities.