Luigi

Luigi is a Python package for building complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization, handling failures and command line integration, and includes Hadoop support. It is intended to stitch together long-running tasks such as Hive, Pig, Hadoop, Spark or database jobs.

GitHub repo Directory import
0 shouts Added Sep 23, 2026 github.com
github.com/spotify/luigi
Luigi website preview

Features

  • handles dependency resolution, workflow management, visualization Source: github.com
  • It also comes with Hadoop support built in. Source: github.com

Links

Tags