Dataset Information

A pilot study to troubleshoot quality control metrics when assessing circulating miRNA expression data reproducibility across study sites.

ABSTRACT:

Background

Given the growing interest in using microRNAs (miRNAs) as biomarkers of early disease, establishment of robust protocols and platforms for miRNA quantification in biological fluids is critical.

Objective

The goal of this multi-center pilot study was to evaluate the reproducibility of NanoString nCounter™ technology when analyzing the abundance of miRNAs in plasma and cystic fluid from patients with pancreatic lesions.

Methods

Using sample triplicates analyzed across three study sites, we assessed potential sources of variability (RNA isolation, sample processing/ligation, hybridization, and lot-to-lot variability) that may contribute to suboptimal reproducibility of miRNA abundance when using nCounter™, and evaluated expression of positive and negative controls, housekeeping genes, spike-in genes, and miRNAs.

Results

Positive controls showed a high correlation across samples from each site (median correlation coefficient, r> 0.9). Most negative control probes had expression levels below background. Housekeeping and spike-in genes each showed a similar distribution of expression and comparable pairwise correlation coefficients of replicate samples across sites. A total of 804 miRNAs showed a similar distribution of pairwise correlation coefficients between replicate samples (p= 0.93). After normalization and selecting miRNAs with expression levels above zero in 80% of samples, 55 miRNAs were identified; heatmap and principal component analysis revealed similar expression patterns and clustering in replicate samples.

Conclusions

Findings from this pilot investigation suggest the nCounter platform can yield reproducible results across study sites. This study underscores the importance of implementing quality control procedures when designing multi-center evaluations of miRNA abundance.

SUBMITTER: Permuth JB

PROVIDER: S-EPMC9428925 | biostudies-literature | 2022

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A pilot study to troubleshoot quality control metrics when assessing circulating miRNA expression data reproducibility across study sites.

Permuth Jennifer B JB Mesa Tania T Williams Sion L SL Cardentey Yoslayma Y Zhang Dongyu D Pawlak Erica A EA Li Jiannong J Cameron Miles E ME Ali Karla N KN Jeong Daniel D Yoder Sean J SJ Chen Dung-Tsa DT Trevino Jose G JG Merchant Nipun N Malafa Mokenge M

Cancer biomarkers : section A of Disease markers 20220101 4

<h4>Background</h4>Given the growing interest in using microRNAs (miRNAs) as biomarkers of early disease, establishment of robust protocols and platforms for miRNA quantification in biological fluids is critical.<h4>Objective</h4>The goal of this multi-center pilot study was to evaluate the reproducibility of NanoString nCounter™ technology when analyzing the abundance of miRNAs in plasma and cystic fluid from patients with pancreatic lesions.<h4>Methods</h4>Using sample triplicates analyzed acr ...[more]

PMID: 35491771

Similar Datasets

Project description:Formalin-fixed, paraffin-embedded (FFPE) tissues have many advantages for identification of risk biomarkers, including wide availability and potential for extended follow-up endpoints. However, RNA derived from archival FFPE samples has limited quality. Here we identified parameters that determine which FFPE samples have the potential for successful RNA extraction, library preparation, and generation of usable RNAseq data. We optimized library preparation protocols designed for use with FFPE samples using seven FFPE and Fresh Frozen replicate pairs, and tested optimized protocols using a study set of 130 FFPE biopsies from women with benign breast disease. Metrics from RNA extraction and preparation procedures were collected and compared with bioinformatics sequencing summary statistics. Finally, a decision tree model was built to learn the relationship between pre-sequencing lab metrics and qc pass/fail status as determined by bioinformatics metrics.. Samples that failed bioinformatics qc tended to have low median sample-wise correlation within the cohort (Spearman correlation < 0.75), low number of reads mapped to gene regions (< 25 million), or low number of detectable genes (11,400 # of detected genes with TPM > 4). The median RNA concentration and pre-capture library Qubit values for qc failed samples were 18.9 ng/ul and 2.08 ng/ul respectively, which were significantly lower than those of qc pass samples (40.8 ng/ul and 5.82 ng/ul). We built a decision tree model based on input RNA concentration, input library qubit values, and achieved an F score of 0.848 in predicting QC status (pass/fail) of FFPE samples. We provide a bioinformatics quality control recommendation for FFPE samples from breast tissue by evaluating bioinformatic and sample metrics. Our results suggest a minimum concentration of 25 ng/ul FFPE-extracted RNA for library preparation and 1.7 ng/ul pre-capture library output to achieve adequate RNA-seq data for downstream bioinformatics analysis.

Project description:BackgroundCorrelation metrics are widely utilized in genomics analysis and often implemented with little regard to assumptions of normality, homoscedasticity, and independence of values. This is especially true when comparing values between replicated sequencing experiments that probe chromatin accessibility, such as assays for transposase-accessible chromatin via sequencing (ATAC-seq). Such data can possess several regions across the human genome with little to no sequencing depth and are thus non-normal with a large portion of zero values. Despite distributed use in the epigenomics field, few studies have evaluated and benchmarked how correlation and association statistics behave across ATAC-seq experiments with known differences or the effects of removing specific outliers from the data. Here, we developed a computational simulation of ATAC-seq data to elucidate the behavior of correlation statistics and to compare their accuracy under set conditions of reproducibility.ResultsUsing these simulations, we monitored the behavior of several correlation statistics, including the Pearson's R and Spearman's [Formula: see text] coefficients as well as Kendall's [Formula: see text] and Top-Down correlation. We also test the behavior of association measures, including the coefficient of determination R[Formula: see text], Kendall's W, and normalized mutual information. Our experiments reveal an insensitivity of most statistics, including Spearman's [Formula: see text], Kendall's [Formula: see text], and Kendall's W, to increasing differences between simulated ATAC-seq replicates. The removal of co-zeros (regions lacking mapped sequenced reads) between simulated experiments greatly improves the estimates of correlation and association. After removing co-zeros, the R[Formula: see text] coefficient and normalized mutual information display the best performance, having a closer one-to-one relationship with the known portion of shared, enhanced loci between simulated replicates. When comparing values between experimental ATAC-seq data using a random forest model, mutual information best predicts ATAC-seq replicate relationships.ConclusionsCollectively, this study demonstrates how measures of correlation and association can behave in epigenomics experiments. We provide improved strategies for quantifying relationships in these increasingly prevalent and important chromatin accessibility assays.

Project description:BackgroundChromatographic peakpicking continues to represent a significant bottleneck in automated LC-MS workflows. Uncontrolled false discovery rates and the lack of manually-calibrated quality metrics require researchers to visually evaluate individual peaks, requiring large amounts of time and breaking replicability. This problem is exacerbated in noisy environmental datasets and for novel separation methods such as hydrophilic interaction columns in metabolomics, creating a demand for a simple, intuitive, and robust metric of peak quality.ResultsHere, we manually labeled four HILIC oceanographic particulate metabolite datasets to assess the performance of individual peak quality metrics. We used these datasets to construct a predictive model calibrated to the likelihood that visual inspection by an MS expert would include a given mass feature in the downstream analysis. We implemented two novel peak quality metrics, a custom signal-to-noise metric and a test of similarity to a bell curve, both calculated from the raw data in the extracted ion chromatogram, and found that these outperformed existing measurements of peak quality. A simple logistic regression model built on two metrics reduced the fraction of false positives in the analysis from 70-80% down to 1-5% and showed minimal overfitting when applied to novel datasets. We then explored the implications of this quality thresholding on the conclusions obtained by the downstream analysis and found that while only 10% of the variance in the dataset could be explained by depth in the default output from the peakpicker, approximately 40% of the variance was explained when restricted to high-quality peaks alone.ConclusionsWe conclude that the poor performance of peakpicking algorithms significantly reduces the power of both univariate and multivariate statistical analyses to detect environmental differences. We demonstrate that simple models built on intuitive metrics and derived from the raw data are more robust and can outperform more complex models when applied to new data. Finally, we show that in properly curated datasets, depth is a major driver of variability in the marine microbial metabolome and identify several interesting metabolite trends for future investigation.

Project description:The monitoring and assessment of data quality is an essential step in the acquisition and analysis of functional MRI (fMRI) data. Ideally data quality monitoring is performed while the data are being acquired and the subject is still in the MRI scanner so that any errors can be caught early and addressed. It is also important to perform data quality assessments at multiple points in the processing pipeline. This is particularly true when analyzing datasets with large numbers of subjects, coming from multiple investigators and/or institutions. These quality control procedures should monitor not only the quality of the original and processed data, but also the accuracy and consistency of acquisition parameters. Between-site differences in acquisition parameters can guide the choice of certain processing steps (e.g., resampling from oblique orientations, spatial smoothing). Various quality control metrics can determine what subjects to exclude from the group analyses, and can also guide additional processing steps that may be necessary. This paper describes a combination of qualitative and quantitative assessments to determine the quality of fMRI data. Processing is performed using the AFNI data analysis package. Qualitative assessments include visual inspection of the structural T1-weighted and fMRI echo-planar images, functional connectivity maps, functional connectivity strength, and temporal signal-to-noise maps concatenated from all subjects into a movie format. Quantitative metrics include the acquisition parameters, statistics about the level of subject motion, temporal signal-to-noise ratio, smoothness of the data, and the average functional connectivity strength. These measures are evaluated at different steps in the processing pipeline to catch gross abnormalities in the data, and to determine deviations in acquisition parameters, the alignment to template space, the level of head motion, and other sources of noise. We also evaluate the effect of different quantitative QC cutoffs, specifically the motion censoring threshold, and the impact of bandpass filtering. These qualitative and quantitative metrics can then provide information about what subjects to exclude and what subjects to examine more closely in the analysis of large datasets.

Dataset Information

A pilot study to troubleshoot quality control metrics when assessing circulating miRNA expression data reproducibility across study sites.

Background

Objective

Methods

Results

Conclusions

Publications

A pilot study to troubleshoot quality control metrics when assessing circulating miRNA expression data reproducibility across study sites.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets