Kedro

Kedro is an open-source Python framework for building reproducible, maintainable, and modular data pipelines and machine-learning projects.

Reviewed by 7wData

On this page

Publisher review

Kedro is an open-source Python framework for building reproducible, maintainable, and modular data pipelines and machine-learning projects. Developed over two years by QuantumBlack (McKinsey's AI division) and released in June 2019, Kedro addresses a persistent challenge in data science: the gap between exploratory notebooks and production-ready code. The framework standardizes workflow through a data catalog, pipeline abstraction, and centralized configuration management, enabling teams to write code that is reproducible, testable, and collaboratively maintainable from day one.

Kedro enforces best practices around code organization and modularity without requiring users to adopt external orchestration platforms—it integrates with Airflow, Prefect, Kubeflow, AWS SageMaker, Databricks, Dask, and Argo for deployment flexibility. Community feedback surfaces a real trade-off: while Kedro excels at long-term team projects requiring version control and structured collaboration, it introduces friction for quick-turnaround consulting work or exploratory data science where notebooks' simplicity is valued. Some practitioners report limited adoption outside McKinsey-led engagements, though the project has grown to 9,000+ GitHub stars and 200+ contributors since being donated to the Linux Foundation in 2022. The framework's VSCode extension and Kedro-Viz visualization tool lower the barrier to adoption, and its learning curve is steeper than notebooks but gentler than full orchestration systems like Dagster.

Get the AI & data signal, daily.

48k+ subscribers read this every morning. One email, both newsletters. Unsubscribe anytime.

How it works

  1. Data Catalog

    Centralized dataset registry that abstracts file paths and formats (CSV, Parquet, HDF5, SQL, cloud storage) so code references datasets by name, not hardcoded paths.

  2. Pipeline & Node Architecture

    Automatic dependency resolution between pure Python functions wrapped as nodes; Kedro infers execution order and parallelization opportunities without explicit DAG definition.

  3. Configuration Management

    External YAML-based parameter and settings files decoupled from code, enabling reproducibility and environment-specific overrides without code changes.

  4. Kedro-Viz Interactive Visualization

    Browser-based graph showing data lineage, execution time, node status, and dataset statistics; recent versions add experiment tracking (scheduled for deprecation in Viz 11.0).

  5. Multi-Platform Deployment

    Seamless integration with Argo Workflows, Prefect, Kubeflow, AWS Batch, AWS SageMaker, Databricks, Dask, and Kubernetes; supports single-machine and distributed execution.

  6. VSCode Extension

    IDE plugin offering code navigation, autocompletion, and inline pipeline visualization for faster development iteration.

  7. Project Templating

    Standardized scaffolding for new projects with pre-configured directory structure, making teams' code organization consistent and reducing onboarding friction.

Strengths and trade-offs

Strengths

  • Enforces modular, testable, version-controllable code that survives team transitions and consulting handoffs better than notebooks.
  • Lower friction than full orchestration platforms (Airflow, Dagster) for teams who prioritize code quality and reproducibility over job scheduling complexity.
  • Well-documented; 2026 Kedro-Viz experiment tracking deprecation shows proactive maintenance and willingness to prune low-adoption features rather than accumulate technical debt.

Trade-offs

  • Steeper learning curve than Jupyter notebooks; requires data scientists to wrap code in pipeline/node abstractions before execution, adding cognitive overhead for exploratory work.
  • Limited adoption outside McKinsey-led projects; some practitioners report few independent use cases in industry, suggesting either niche product-market fit or slow grassroots adoption.
  • Data catalog design assumes structured, regularly accessed datasets; struggles with unstructured, one-off analyses or manual file movement workflows common in consulting.

Pricing context

Kedro is free and open-source under the Apache 2.0 license, hosted by the Linux Foundation's AI & Data program. There are no licensing fees or usage tiers. Optional third-party support services (e.g., AWS Marketplace repackagings) may charge for installation and maintenance assistance, but the core framework carries no cost.

Alternatives

User reviews

No user reviews yet. Be the first to write one.

Sources

Reporting on this tool draws on these publicly available sources.

  1. kedro.org — Official Kedro website confirming open-source status, key features (Data Catalog, Kedro-Viz, pipeline abstraction, deployment options)
  2. www.mckinsey.com — McKinsey's official announcement of donating Kedro to Linux Foundation; confirms creation by QuantumBlack (McKinsey's AI division)
  3. medium.com — QuantumBlack's original announcement (June 2019); documents two years of internal development across 50+ projects before open-source release; explains problem Kedro solves (notebook-to-production gap)
  4. news.ycombinator.com — Community feedback on trade-offs: productivity impact on notebooks, onboarding friction in consulting, limited real-world adoption, learning curve concerns
  5. www.kdnuggets.com — Feature overview, use cases (prototype-to-production conversion, reproducibility, team collaboration), strengths in modularity and structure
  6. dagster.io — Comparative analysis positioning Kedro as emphasizing code organization and reproducibility; 9,000+ GitHub stars, 200+ contributors