Optimized distributed systems achieve significant performance improvement on sorted merging of massive VCF files.

ABSTRACT:

Background

Sorted merging of genomic data is a common data operation necessary in many sequencing-based studies. It involves sorting and merging genomic data from different subjects by their genomic locations. In particular, merging a large number of variant call format (VCF) files is frequently required in large-scale whole-genome sequencing or whole-exome sequencing projects. Traditional single-machine based methods become increasingly inefficient when processing large numbers of files due to the excessive computation time and Input/Output bottleneck. Distributed systems and more recent cloud-based systems offer an attractive solution. However, carefully designed and optimized workflow patterns and execution plans (schemas) are required to take full advantage of the increased com

SUBMITTER: Sun X 

PROVIDER: S-EPMC6007233 | biostudies-literature | 2018 Jun

REPOSITORIES: biostudies-literature