Use law to build complex and large-scale task workflows.
copied from cf-staging / lawUse law to build complex and large-scale task workflows. It is build on top of luigi and adds abstractions for run locations, storage locations and software environments. Law strictly disentangles these building blocks and ensures they remain interchangeable and resource-opportunistic.
Key features: - CLI with auto-completion and interactive status and dependency inspection. - Remote targets with automatic retries and local caching: WebDAV, HTTP, Dropbox, SFTP, all WLCG protocols (srm, xrootd, rfio, dcap, gsiftp, ...) - Automatic submission to batch systems from within tasks: HTCondor, LSF, gLite, ARC - Environment sandboxing, configurable on task level: Docker, Singularity, Sub-Shells
