Dataset Information

False-positive and false-negative risks for individual multicentre trials in critical care.

ABSTRACT:

Background

In medical research, null hypothesis significance testing (NHST) is the dominant framework for statistical inference. NHST involves calculating P-values and confidence intervals to quantify the evidence against the null hypothesis of no effect. However, P-values and confidence intervals cannot tell us the probability that the hypothesis is true. In contrast, false-positive risk (FPR) and false-negative risk (FNR) are post-test probabilities concerning the truth of the hypothesis, that is to say, the probability a real effect exists.

Methods

We calculated the FPR or FNR for 53 individual multicentre trials in critical care based on a pretest probability of 0.5 that the hypothesis was true.

Results

For trials reporting statistical significance, the FPR varied between 0.1% and 57.6%. For trials reporting non-significance, the FNR varied between 1.7% and 36.9%. Twenty-six of 47 trials (55.3%) reporting non-significance provided strong or very strong evidence in favour of the null hypothesis; the remaining trials provided limited evidence. There was no obvious relationship between the P-value and the FNR.

Conclusions

The FPR and FNR showed marked variability, indicating that the probability of a real or absent treatment effect differed substantially between trials. Only one trial reporting statistical significance provided convincing evidence of a real treatment effect, and nearly half of all trials reporting non-significance provided limited evidence for the absence of a treatment effect. Our findings suggest that the quality of evidence from multicentre trials in critical care is highly variable.

SUBMITTER: Sidebotham D

PROVIDER: S-EPMC10430847 | biostudies-literature | 2022 Mar

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

False-positive and false-negative risks for individual multicentre trials in critical care.

Sidebotham David D Barlow C Jake CJ

BJA open 20220301

<h4>Background</h4>In medical research, null hypothesis significance testing (NHST) is the dominant framework for statistical inference. NHST involves calculating <i>P</i>-values and confidence intervals to quantify the evidence against the null hypothesis of no effect. However, <i>P</i>-values and confidence intervals cannot tell us the probability that the hypothesis is true. In contrast, false-positive risk (FPR) and false-negative risk (FNR) are post-test probabilities concerning the truth o ...[more]

PMID: 37588693

Similar Datasets

Project description:Objectives: With the worldwide spread of coronavirus disease 2019 (COVID-19) caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), various antibody detection kits have been developed to test for SARS-CoV-2- specific IgG, IgM, and total antibody. However, the use of different testing methods under various heat-inactivation conditions might affect the COVID-19 detection results. Methods: Seven different antibody detection kits produced by four manufacturers for detection of SARS-CoV-2 IgG, IgM, and total antibody were tested at Wuhan Huoshenshan Hospital, China. Most of the kits used the indirect immunity, capture, and double-antigen sandwich methods. The effects of various heat-inactivation conditions on SARS-CoV-2-specific IgG, IgM, and total antibody detection were analyzed for the different test methods. Results: Using the indirect immunity method, values for SARS-CoV-2 IgG antibody significantly increased and those for IgM antibody decreased with increasing temperature of heat-inactivation using indirect immunity method. However, values for SARS-CoV-2 IgM and total antibody showed no change when the capture and double-antigen sandwich methods were used. The changes in IgG and IgM antibody values with the indirect immunity method indicated that heat-inactivation could affect COVID-19 detection results obtained using this method. In particular, 18 (22.2%) SARS-CoV-2 IgM positive samples were detected as negative with heat-inactivation at 65°C for 30 min, and one (25%) IgG negative sample was detected as positive after heat-inactivation at 56°C for 60 min and 60°C for 30 min. Conclusions: Heat-inactivation could increase SARS-CoV-2 IgG antibody values, and decrease IgM antibody values, causing potential false-positive or false-negative results for COVID-19 antibody detection using the indirect immunity method. Thus, before conducting antibody testing, the testing platforms should be evaluated in accordance with the relevant requirements to ensure accurate COVID-19 detection results.

Project description:The massive sequencing of transposon insertion mutant libraries (Tn-Seq) represents a commonly used method to determine essential genes in bacteria. Using a hypersaturated transposon mutant library consisting of 400,096 unique Tn insertions, 523 genes were classified as essential in Escherichia coli K-12 MG1655. This provided a useful genome-wide gene essentiality landscape for rapidly identifying 233 of 301 essential genes previously validated by a knockout study. However, there was a discrepancy in essential gene sets determined by conventional gene deletion methods and Tn-Seq, although different Tn-Seq studies reported different extents of discrepancy. We have elucidated two causes of this discrepancy. First, 68 essential genes not detected by Tn-Seq contain nonessential subgenic domains that are tolerant to transposon insertion, which leads to the false assignment of an essential gene as a nonessential or dispensable gene. These genes exhibited a high level of transposon insertion in their subgenic nonessential domains. In contrast, 290 genes were additionally categorized as essential by Tn-Seq, although their knockout mutants were available. The comparative analysis of Tn-Seq and high-resolution footprinting of nucleoid-associated proteins (NAPs) revealed that a protein-DNA interaction hinders transposon insertion. We identified 213 false-positive genes caused by NAP-genome interactions. These two limitations have to be considered when addressing essential bacterial genes using Tn-Seq. Furthermore, a comparative analysis of high-resolution Tn-Seq with other data sets is required for a more accurate determination of essential genes in bacteria. IMPORTANCE Transposon mutagenesis is an efficient way to explore gene essentiality of a bacterial genome. However, there was a discrepancy between the essential gene set determined by transposon mutagenesis and that determined using single-gene knockout strains. In this study, we generated a hypersaturated Escherichia coli transposon mutant library comprising approximately 400,000 different mutants. Determination of transposon insertion sites using next-generation sequencing provided a high-resolution essentiality landscape of the E. coli genome. We identified false negatives of essential gene discovery due to the permissive insertion of transposons in the C-terminal region. Comparisons between the transposon insertion landscape with binding profiles of DNA-binding proteins revealed interference of nucleoid-associated proteins to transposon insertion, generating false positives of essential gene discovery. Consideration of these findings is required to avoid the misinterpretation of transposon mutagenesis results.

Project description:BackgroundNext-generation sequencing pipelines often perform error correction as a preprocessing step to obtain cleaned input data. State-of-the-art error correction programs are able to reliably detect and correct the majority of sequencing errors. However, they also introduce new errors by making false-positive corrections. These correction mistakes can have negative impact on downstream analysis, such as k-mer statistics, de-novo assembly, and variant calling. This motivates the need for more precise error correction tools.ResultsWe present CARE 2.0, a context-aware read error correction tool based on multiple sequence alignment targeting Illumina datasets. In addition to a number of newly introduced optimizations its most significant change is the replacement of CARE 1.0's hand-crafted correction conditions with a novel classifier based on random decision forests trained on Illumina data. This results in up to two orders-of-magnitude fewer false-positive corrections compared to other state-of-the-art error correction software. At the same time, CARE 2.0 is able to achieve high numbers of true-positive corrections comparable to its competitors. On a simulated full human dataset with 914M reads CARE 2.0 generates only 1.2M false positives (FPs) (and 801.4M true positives (TPs)) at a highly competitive runtime while the best corrections achieved by other state-of-the-art tools contain at least 3.9M FPs and at most 814.5M TPs. Better de-novo assembly and improved k-mer analysis show the applicability of CARE 2.0 to real-world data.ConclusionFalse-positive corrections can negatively influence down-stream analysis. The precision of CARE 2.0 greatly reduces the number of those corrections compared to other state-of-the-art programs including BFC, Karect, Musket, Bcool, SGA, and Lighter. Thus, higher-quality datasets are produced which improve k-mer analysis and de-novo assembly in real-world datasets which demonstrates the applicability of machine learning techniques in the context of sequencing read error correction. CARE 2.0 is written in C++/CUDA for Linux systems and can be run on the CPU as well as on CUDA-enabled GPUs. It is available at https://github.com/fkallen/CARE .

Project description:BackgroundVector-borne diseases are major public health concerns worldwide. For many of them, vector control is still key to primary prevention, with control actions planned and evaluated using vector occurrence records. Yet vectors can be difficult to detect, and vector occurrence indices will be biased whenever spurious detection/non-detection records arise during surveys. Here, we investigate the process of Chagas disease vector detection, assessing the performance of the surveillance method used in most control programs--active triatomine-bug searches by trained health agents.Methodology/principal findingsControl agents conducted triplicate vector searches in 414 man-made ecotopes of two rural localities. Ecotope-specific 'detection histories' (vectors or their traces detected or not in each individual search) were analyzed using ordinary methods that disregard detection failures and multiple detection-state site-occupancy models that accommodate false-negative and false-positive detections. Mean (± SE) vector-search sensitivity was ∼ 0.283 ± 0.057. Vector-detection odds increased as bug colonies grew denser, and were lower in houses than in most peridomestic structures, particularly woodpiles. False-positive detections (non-vector fecal streaks misidentified as signs of vector presence) occurred with probability ∼ 0.011 ± 0.008. The model-averaged estimate of infestation (44.5 ± 6.4%) was ∼ 2.4-3.9 times higher than naïve indices computed assuming perfect detection after single vector searches (11.4-18.8%); about 106-137 infestation foci went undetected during such standard searches.Conclusions/significanceWe illustrate a relatively straightforward approach to addressing vector detection uncertainty under realistic field survey conditions. Standard vector searches had low sensitivity except in certain singular circumstances. Our findings suggest that many infestation foci may go undetected during routine surveys, especially when vector density is low. Undetected foci can cause control failures and induce bias in entomological indices; this may confound disease risk assessment and mislead program managers into flawed decision making. By helping correct bias in naïve indices, the approach we illustrate has potential to critically strengthen vector-borne disease control-surveillance systems.

Dataset Information

False-positive and false-negative risks for individual multicentre trials in critical care.

Background

Methods

Results

Conclusions

Publications

False-positive and false-negative risks for individual multicentre trials in critical care.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets