Dataset Information

FastqCleaner: an interactive Bioconductor application for quality-control, filtering and trimming of FASTQ files.

ABSTRACT:

Background

Exploration and processing of FASTQ files are the first steps in state-of-the-art data analysis workflows of Next Generation Sequencing (NGS) platforms. The large amount of data generated by these technologies has put a challenge in terms of rapid analysis and visualization of sequencing information. Recent integration of the R data analysis platform with web visual frameworks has stimulated the development of user-friendly, powerful, and dynamic NGS data analysis applications.

Results

This paper presents FastqCleaner, a Bioconductor visual application for both quality-control (QC) and pre-processing of FASTQ files. The interface shows diagnostic information for the input and output data and allows to select a series of filtering and trimming operations in an interactive framework. FastqCleaner combines the technology of Bioconductor for NGS data analysis with the data visualization advantages of a web environment.

Conclusions

FastqCleaner is an user-friendly, offline-capable tool that enables access to advanced Bioconductor infrastructure. The novel concept of a Bioconductor interactive application that can be used without the need for programming skills, makes FastqCleaner a valuable resource for NGS data analysis.

SUBMITTER: Roser LG

PROVIDER: S-EPMC6599294 | biostudies-literature | 2019 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

FastqCleaner: an interactive Bioconductor application for quality-control, filtering and trimming of FASTQ files.

Roser Leandro Gabriel LG Agüero Fernán F Sánchez Daniel Oscar DO

BMC bioinformatics 20190628 1

<h4>Background</h4>Exploration and processing of FASTQ files are the first steps in state-of-the-art data analysis workflows of Next Generation Sequencing (NGS) platforms. The large amount of data generated by these technologies has put a challenge in terms of rapid analysis and visualization of sequencing information. Recent integration of the R data analysis platform with web visual frameworks has stimulated the development of user-friendly, powerful, and dynamic NGS data analysis applications ...[more]

PMID: 31253077

Similar Datasets

Project description:BackgroundSome applications, especially those clinical applications requiring high accuracy of sequencing data, usually have to face the troubles caused by unavoidable sequencing errors. Several tools have been proposed to profile the sequencing quality, but few of them can quantify or correct the sequencing errors. This unmet requirement motivated us to develop AfterQC, a tool with functions to profile sequencing errors and correct most of them, plus highly automated quality control and data filtering features. Different from most tools, AfterQC analyses the overlapping of paired sequences for pair-end sequencing data. Based on overlapping analysis, AfterQC can detect and cut adapters, and furthermore it gives a novel function to correct wrong bases in the overlapping regions. Another new feature is to detect and visualise sequencing bubbles, which can be commonly found on the flowcell lanes and may raise sequencing errors. Besides normal per cycle quality and base content plotting, AfterQC also provides features like polyX (a long sub-sequence of a same base X) filtering, automatic trimming and K-MER based strand bias profiling.ResultsFor each single or pair of FastQ files, AfterQC filters out bad reads, detects and eliminates sequencer's bubble effects, trims reads at front and tail, detects the sequencing errors and corrects part of them, and finally outputs clean data and generates HTML reports with interactive figures. AfterQC can run in batch mode with multiprocess support, it can run with a single FastQ file, a single pair of FastQ files (for pair-end sequencing), or a folder for all included FastQ files to be processed automatically. Based on overlapping analysis, AfterQC can estimate the sequencing error rate and profile the error transform distribution. The results of our error profiling tests show that the error distribution is highly platform dependent.ConclusionMuch more than just another new quality control (QC) tool, AfterQC is able to perform quality control, data filtering, error profiling and base correction automatically. Experimental results show that AfterQC can help to eliminate the sequencing errors for pair-end sequencing data to provide much cleaner outputs, and consequently help to reduce the false-positive variants, especially for the low-frequency somatic mutations. While providing rich configurable options, AfterQC can detect and set all the options automatically and require no argument in most cases.

Project description:This repository contains all the FASTQ files for the five data modalities (scRNA-seq, scATAC-seq, Multiome, CITE-seq+scVDJ-seq, and spatial transcriptomics) used in the article \\"An Atlas of Cells in The Human Tonsil,\\" published in Immunity in 2024. Inspired by the TCGA barcodes, we have named each fastq file with the following convention: [TECHNOLOGY].[DONOR_ID].[SUBPROJECT].[GEM_ID].[LIBRARY_ID].[LIBRARY_TYPE].[LANE].[READ].fastq.gz which allows to retrieve all metadata from the name itself. Here is a full description of each field: - TECHNOLOGY: scRNA-seq, scATAC-seq, Multiome, CITE-seq+scVDJ-seq, and spatial transcriptomics (Visium). We also include the fastq files associated with the multiome experiments performed on two mantle cell lymphoma patients (MCL). - DONOR_ID: identifier for each of the 17 patients included in the cohort. We provide the donor-level metadata in the file \\"tonsil_atlas_donor_metadata.csv\\", including the hospital, sex, age, age group, cause for tonsillectomy and cohort type for every donor. - SUBPROJECT: each subproject corresponds to one run of the 10x Genomics Chromium™ Chip. - GEM_ID: each run of the 10x Genomics Chromium™ Chip consists of up to 8 \\"GEM wells\\" (see https://www.10xgenomics.com/support/software/cell-ranger/getting-started/cr-glossary): a set of partitioned cells (Gel Beads-in-emulsion) from a single 10x Genomics Chromium™ Chip channel. We give a unique identifier to each of these channels. - LIBRARY_ID: one or more sequencing libraries can be derived from a GEM well. For instance, multiome yields two libraries (ATAC and RNA) and CITE-seq+scVDJ yields 4 libraries (RNA, ADT, BCR, TCR). - LIBRARY_TYPE: the type of library for each library_id. Note that we used cell hashing () for a subset of the scRNA-seq libraries, and thus the library_type can be \\"not_hashed\\", \\"hashed_cdna\\" (RNA expression) or \\"hashed_hto\\" (the hashtag oligonucleotides). - LANE: to increase sequencing depth, each library was sequenced in more than one lane. Important: all lanes corresponding to the same sequencing library need to be inputed together to cellranger, because they come from the same set of cells. - READ: for scATAC-seq we have three reads (R1, R2 or R3), see cellranger-atac's documentation. While we find these names to be the most useful, they need to be changed to follow cellranger's conventions. We provide a code snippet in the README file of the GitHub repository associated with the tonsil atlas to convert between both formats (https://github.com/Single-Cell-Genomics-Group-CNAG-CRG/TonsilAtlas/). Besides the fastq files, cellranger (and other mappers) require additional files, which we also provide in this repository: - cell_hashing_metadata.csv: as mentioned above, we ran cell hashing (10.1186/s13059-018-1603-1) to detect doublets and reduce cost per cell. This file provides the sequence of the hashtag oligonucleotides in cellranger convention to allow demultiplexing. - cite_seq_feature_reference.csv: similar to the previous file, this one links each protein surface marker to the hashtag oligonucleotide that identified it in the CITE-seq experiment. - V10M16-059.gpr and V19S23-039.gpr: these correspond to the two slides of the two Visium experiments performed in the tonsil atlas. They are needed to run spaceranger. - [GEM_ID]_[SLIDE]_[CAPTURE_AREA].jpg: 8 images associated with the Visium experiments. Here, GEM_ID refers to each of the 4 capture areas in each slide. - [TECHNOLOGY]_sequencing_metadata.csv: the GEM-level metadata for each technology. It includes the relationship between subproject, gem_id, library_id, library_type and donor_id. These are the other repositories associated with the tonsil atlas: - Expression and accessibility matrices: https://zenodo.org/records/10373041 - Seurat objects: https://zenodo.org/records/8373756 - HCATonsilData package: https://bioconductor.org/packages/release/data/experiment/html/HCATonsilData.html - Azimuth: https://azimuth.hubmapconsortium.org/ - Github: https://github.com/Single-Cell-Genomics-Group-CNAG-CRG/TonsilAtlas

Project description:As sequencing becomes more accessible, there is an acute need for novel compression methods to efficiently store sequencing files. Omics analytics can leverage sequencing technologies to enhance biomedical research and individualize patient care, but sequencing files demand immense storage capabilities, particularly when sequencing is utilized for longitudinal studies. Addressing the storage challenges posed by these technologies is crucial for omics analytics to achieve their full potential. We present a novel lossless, reference-free compression algorithm, GeneSqueeze, that leverages the patterns inherent in the underlying components of FASTQ files to solve this need. GeneSqueeze's benefits include an auto-tuning compression protocol based on each file's distribution, lossless preservation of IUPAC nucleotides and read identifiers, and unrestricted FASTQ/A file attributes (i.e., read length, number of reads, or read identifier format). We compared GeneSqueeze to the general-purpose compressor, gzip, and to a domain-specific compressor, SPRING, to assess performance. Due to GeneSqueeze's current Python implementation, GeneSqueeze underperformed as compared to gzip and SPRING in the time domain. GeneSqueeze and gzip achieved 100% lossless compression across all elements of the FASTQ files (i.e. the read identifier, sequence, quality score and ' + ' lines). GeneSqueeze and gzip compressed all files losslessly, while both SPRING's traditional and lossless modes exhibited data loss of non-ACGTN IUPAC nucleotides and of metadata following the ' + ' on the separator line. GeneSqueeze showed up to three times higher compression ratios as compared to gzip, regardless of read length, number of reads, or file size, and had comparable compression ratios to SPRING across a variety of factors. Overall, GeneSqueeze represents a competitive and specialized compression method for FASTQ/A files containing nucleotide sequences. As such, GeneSqueeze has the potential to significantly reduce the storage and transmission costs associated with large omics datasets without sacrificing data integrity.

Project description:BackgroundThe increasingly widespread use of next generation sequencing protocols has brought the need for the development of user-friendly raw data processing tools. Here, we explore 2FAST2Q, a versatile and intuitive standalone program capable of extracting and counting feature occurrences in FASTQ files. Despite 2FAST2Q being previously described as part of a CRISPRi-seq analysis pipeline, in here we further elaborate on the program's functionality, and its broader applicability and functions.Methods2FAST2Q is built in Python, with published standalone executables in Windows MS, MacOS, and Linux. It has a familiar user interface, and uses an advanced custom sequence searching algorithm.ResultsUsing published CRISPRi datasets in which Escherichia coli and Mycobacterium tuberculosis gene essentiality, as well as host-cell sensitivity towards SARS-CoV2 infectivity were tested, we demonstrate that 2FAST2Q efficiently recapitulates published output in read counts per provided feature. We further show that 2FAST2Q can be used in any experimental setup that requires feature extraction from raw reads, being able to quickly handle Hamming distance based mismatch alignments, nucleotide wise Phred score filtering, custom read trimming, and sequence searching within a single program. Moreover, we exemplify how different FASTQ read filtering parameters impact downstream analysis, and suggest a default usage protocol. 2FAST2Q is easier to use and faster than currently available tools, efficiently processing not only CRISPRi-seq / random-barcode sequencing datasets on any up-to-date laptop, but also handling the advanced extraction of de novo features from FASTQ files. We expect that 2FAST2Q will not only be useful for people working in microbiology but also for other fields in which amplicon sequencing data is generated. 2FAST2Q is available as an executable file for all current operating systems without installation and as a Python3 module on the PyPI repository (available at https://veeninglab.com/2fast2q).

Dataset Information

FastqCleaner: an interactive Bioconductor application for quality-control, filtering and trimming of FASTQ files.

Background

Results

Conclusions

Publications

FastqCleaner: an interactive Bioconductor application for quality-control, filtering and trimming of FASTQ files.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets