Streammd: fast low-memory duplicate marking using a Bloom filter.

ABSTRACT:

Summary

Identification of duplicate templates is a common preprocessing step in bulk sequence analysis; for large libraries, this can be resource intensive. Here, we present streammd: a fast, memory-efficient, single-pass duplicate marker operating on the principle of a Bloom filter. streammd closely reproduces outputs from Picard MarkDuplicates while being substantially faster, and requires much less memory than SAMBLASTER.

Availability and implementation

streammd is a C++ program available from GitHub https://github.com/delocalizer/streammd under the MIT license.

SUBMITTER: Leonard C 

PROVIDER: S-EPMC10112951 | biostudies-literature | 2023 Apr

REPOSITORIES: biostudies-literature