Build complex pipelines of batch jobs
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization, handling failures, command line integration, and much more. It also comes with Hadoop support built in.
