lakeFS

lakeFS is an open-source version control system for data lakes that brings Git-like semantics to data management.

Reviewed by 7wData

On this page

Publisher review

lakeFS is an open-source version control system for data lakes that brings Git-like semantics to data management. Built by Treeverse (founded 2020 by Oz Katz and Dr. Einat Orr), lakeFS operates as a metadata layer on top of cloud object storage (S3, Azure Blob, GCS) and S3-compatible systems, enabling teams to branch, merge, commit, revert, and tag datasets without copying data.

The system is designed for teams managing data pipelines and ML experiments at scale—it works transparently with Spark, Trino, Databricks, and table formats like Delta Lake, Iceberg, and Hudi. A branch operation creates a logical copy with zero-copy semantics; an atomic commit snapshots the entire state; reverting rolls data back exactly. lakeFS is freely available as open-source (Apache 2.0) and supports self-hosted or cloud deployment. Enterprise versions add SSO, RBAC, audit logs, and SLA support, though specific pricing is not publicly listed.

The tool has been recognized as a Representative Vendor in the 2025 Gartner Market Guide for DataOps Tools, signaling maturity in the data operations space. Trade-offs include a metadata control plane that becomes a single point of coordination (requiring operational reliability), and diffing performance that scales linearly with dataset size rather than change set size. Adoption is highest among data-forward organizations with reproducibility and governance needs; teams with ad-hoc data cultures may find the structure constraining.

Get the AI & data signal, daily.

48k+ subscribers read this every morning. One email, both newsletters. Unsubscribe anytime.

How it works

  1. Git-style versioning

    Branch, merge, commit, and revert data with atomic operations that snapshot state without copying objects.

  2. Zero-copy branching

    Create isolated data versions as metadata-only operations; branching costs scale with metadata size, not data volume.

  3. S3-compatible API gateway

    Present repositories as S3 buckets so existing tools (Spark, Trino, Athena, Presto) integrate without refactoring.

  4. Format-agnostic integration

    Works with Parquet, Avro, CSV, Iceberg, Delta Lake, and Hudi; enforces standards across mixed-format estates.

  5. RBAC and access audit

    Role-based controls, SSO, SCIM, and immutable audit logs for compliance; governance features bundled in Enterprise tier.

  6. Configurable garbage collection

    Reclaim storage from abandoned branches and unreferenced objects; prevents runaway metadata bloat in long-lived repositories.

  7. Pre-commit hooks and data quality gates

    Attach tests and validations to commits; catch data anomalies before promotion to production without duplicating datasets.

Strengths and trade-offs

Strengths

  • Zero-copy branching dramatically reduces storage costs compared to data copying approaches; Git semantics are familiar to engineering teams managing code and data in tandem.
  • Transparent to existing compute engines (Spark, Trino, Databricks) via S3 API; no tool rewrites or data migration required for adoption.
  • Immutable audit trail and RBAC make compliance reporting straightforward; atomic commits guarantee consistency across multi-object changes.

Trade-offs

  • Metadata control plane is a coordination bottleneck; if the lakeFS server is unavailable, the data lake becomes read-only despite live underlying storage.
  • Diffing between versions is O(n) in dataset size because commits store full repository state as JSON arrays; large repositories see sluggish diff performance.
  • Enterprise pricing is opaque and custom-quoted; no published per-user costs or volume tiers, so budgeting requires sales engagement early.

Pricing context

lakeFS is free and open-source under Apache 2.0 for self-hosted deployments. A managed lakeFS Cloud option is available but specific tier details are not published. Enterprise adds RBAC, SSO, SCIM, audit logs, Iceberg REST Catalog, metadata search, transactional mirroring, and SOC2 compliance, with custom pricing requiring direct contact with sales (support@treeverse.io). No minimum seats or per-user licensing is disclosed; Enterprise contracts are tailored to organization size and storage scale.

Alternatives

User reviews

No user reviews yet. Be the first to write one.

Sources

Reporting on this tool draws on these publicly available sources.

  1. lakefs.io — Core product description, key features (data quality, reproducibility, governance, integration), Gartner 2025 DataOps recognition
  2. docs.lakefs.io — Technical architecture (version control operations, zero-copy branching, S3 API compatibility, storage backend support)
  3. lakefs.io — Founding year (2020), founders (Oz Katz and Dr. Einat Orr), company mission
  4. github.com — Open-source license (Apache 2.0), project maturity (5.4k GitHub stars), Treeverse as maintainer
  5. lakefs.io — Pricing tiers (Open Source free, Enterprise custom), feature differences (SSO, RBAC, SCIM, audit logs, governance capabilities)
  6. lakefs.io — Integration with Iceberg, Delta Lake, and Hudi table formats; format-agnostic positioning
  7. sider.ai — Trade-offs and weaknesses including metadata control plane bottleneck and diff performance limitations
  8. lakefs.io — Integration with Databricks, S3A filesystem compatibility, real-world deployment patterns