Pachyderm

Pachyderm is a project for data-centric pipelines and data versioning. It provides parallelized processing of multi-stage, language-agnostic pipelines with data versioning and data lineage tracking, and pipelines trigger automatically when data changes. It runs on Kubernetes and uses standard object stores for storage.

GitHub repo Directory import
0 shouts Added Sep 23, 2026 github.com
github.com/pachyderm/pachyderm
Pachyderm website preview

Features

  • Data-driven pipelines automatically trigger based on detecting data changes Source: github.com

Platforms

  • Autoscaling and parallel processing built on Kubernetes for resource orchestration Source: github.com

Links

Tags