CWL-Airflow: a lightweight pipeline manager supporting Common Workflow Language.

ABSTRACT:

Background

Massive growth in the amount of research data and computational analysis has led to increased use of pipeline managers in biomedical computational research. However, each of the >100 such managers uses its own way to describe pipelines, leading to difficulty porting workflows to different environments and therefore poor reproducibility of computational studies. For this reason, the Common Workflow Language (CWL) was recently introduced as a specification for platform-independent workflow description, and work began to transition existing pipelines and workflow managers to CWL.

Findings

Herein, we present CWL-Airflow, a package that adds support for CWL to the Apache Airflow pipeline manager. CWL-Airflow uses CWL version 1.0 specification and can run workflows on s

SUBMITTER: Kotliar M 

PROVIDER: S-EPMC6639121 | biostudies-literature | 2019 Jul

REPOSITORIES: biostudies-literature