BigSeqKit: a parallel Big Data toolkit to process FASTA and FASTQ files at scale.

ABSTRACT:

Background

High-throughput sequencing technologies have led to an unprecedented explosion in the amounts of sequencing data available, which are typically stored using FASTA and FASTQ files. We can find in the literature several tools to process and manipulate those type of files with the aim of transforming sequence data into biological knowledge. However, none of them are well fitted for processing efficiently very large files, likely in the order of terabytes in the following years, since they are based on sequential processing. Only some routines of the well-known seqkit tool are partly parallelized. In any case, its scalability is limited to use few threads on a single computing node.

Results

Our approach, BigSeqKit, takes advantage of a high-performance computing-Big D

SUBMITTER: Pineiro C 

PROVIDER: S-EPMC10388699 | biostudies-literature | 2022 Dec

REPOSITORIES: biostudies-literature