Anticipating missing reference standard data when planning diagnostic accuracy studies.
Ontology highlight
ABSTRACT:
SUBMITTER: Naaktgeboren CA
PROVIDER: S-EPMC4772780 | biostudies-literature | 2016 Feb
REPOSITORIES: biostudies-literature
Publications
Anticipating missing reference standard data when planning diagnostic accuracy studies.
Naaktgeboren Christiana A CA de Groot Joris A H JA Rutjes Anne W S AW Bossuyt Patrick M M PM Reitsma Johannes B JB Moons Karel G M KG
BMJ (Clinical research ed.) 20160209
Similar Datasets
Project description:BackgroundStaquet et al. and Brenner both developed correction methods to estimate the sensitivity and specificity of a binary-response index test when the reference standard is imperfect and its sensitivity and specificity are known. However, to our knowledge, no study has compared the statistical properties of these methods, despite their long application in diagnostic accuracy studies.AimTo compare the correction methods developed by Staquet et al. and Brenner.MethodsSimulations techniques were employed to compare the methods under assumptions that the new test and the reference standard are conditionally independent or dependent given the true disease status of an individual. Three clinical datasets were analysed to understand the impact of using each method to inform clinical decision-making.ResultsUnder the assumption of conditional independence, the Staquet et al. correction method outperforms the Brenner correction method irrespective of the prevalence of disease and whether the performance of the reference standard is better or worse than the index test. However, when the prevalence of the disease is high (> 0.9) or low (< 0.1), the Staquet et al. correction method can produce illogical results (i.e. results outside [0,1]). Under the assumption of conditional dependence; both methods failed to estimate the sensitivity and specificity of the index test especially when the covariance terms between the index test and the reference standard is not close to zero.ConclusionWhen the new test and the imperfect reference standard are conditionally independent, and the sensitivity and specificity of the imperfect reference standard are known, the Staquet et al. correction method outperforms the Brenner method. However, where the prevalence of the target condition is very high or low or the two tests are conditionally dependent, other statistical methods such as latent class approaches should be considered.
| S-EPMC8040223 | biostudies-literature
Project description:PurposeTo retrospectively evaluate diagnostic performance of dual-energy subtraction radiography (DESR) for interpretation of chest radiographs compared to conventional radiography (CR) using computed tomography (CT) as standard of reference.Material and methodsA total of 199 patients (75 female, median age 67) were included in this institutional review board (IRB)-approved clinical trial. All patients were scanned in posteroanterior and lateral direction with dual-shot DE-technique. Chest CT was performed within ±72 hours. The system provides three types of images: bone weighted-image, soft tissue weighted-image, herein termed as DESR-images, and a standard image, termed CR-image (marked as CR-image). Images were evaluated by two radiologists for presence of inserted life support lines, pneumothorax, pleural effusion, infectious consolidation, interstitial lung changes, tumor, skeletal alterations, soft tissue alterations, aortic or tracheal calcification and pleural thickening. Inter-observer agreement between readers and diagnostic performance were calculated. McNemar's test was used to test for significant differences.ResultsMean inter-observer agreement throughout the investigated parameters was higher in DESR images compared to CR-images (kDESR = 0.935 vs. kCR = 0.858). DESR images provided significantly increased sensitivity compared to CR-images for the detection of infectious consolidations (42% vs. 62%), tumor (46% vs. 57%), interstitial lung changes (69% vs. 87%) and aortic or tracheal calcification (25 vs. 73%) (p<0.05). There were no significant differences in sensitivity for the detection of inserted life support lines, pneumothorax, pleural effusion, skeletal alterations, soft tissue alterations or pleural thickening (p>0.05).ConclusionDESR increases significantly the sensibility without affecting the specificity evaluating chest radiographs, with emphasis on the detection of interstitial lung diseases.
| S-EPMC5354458 | biostudies-literature
Project description:Several statistical methods for meta-analysis of diagnostic accuracy studies have been discussed in the presence of a gold standard. However, in practice, the selected reference test may be imperfect due to measurement error, non-existence, invasive nature, or expensive cost of a gold standard. It has been suggested that treating an imperfect reference test as a gold standard can lead to substantial bias in the estimation of diagnostic test accuracy. Recently, two models have been proposed to account for imperfect reference test, namely, a multivariate generalized linear mixed model (MGLMM) and a hierarchical summary receiver operating characteristic (HSROC) model. Both models are very flexible in accounting for heterogeneity in accuracies of tests across studies as well as the dependence between tests. In this article, we show that these two models, although with different formulations, are closely related and are equivalent in the absence of study-level covariates. Furthermore, we provide the exact relations between the parameters of these two models and assumptions under which two models can be reduced to equivalent submodels. On the other hand, we show that some submodels of the MGLMM do not have corresponding equivalent submodels of the HSROC model, and vice versa. With three real examples, we illustrate the cases when fitting the MGLMM and HSROC models leads to equivalent submodels and hence identical inference, and the cases when the inferences from two models are slightly different. Our results generalize the important relations between the bivariate generalized linear mixed model and HSROC model when the reference test is a gold standard.
| S-EPMC4416105 | biostudies-literature
Project description:Novel typhoid diagnostics currently under development have the potential to improve clinical care, surveillance, and the disease burden estimates that support vaccine introduction. Blood culture is most often used as the reference method to evaluate the accuracy of new typhoid tests; however, it is recognized to be an imperfect gold standard. If no single gold standard test exists, use of a composite reference standard (CRS) can improve estimation of diagnostic accuracy. Numerous studies have used a CRS to evaluate new typhoid diagnostics; however, there is no consensus on an appropriate CRS. In order to evaluate existing tests for use as a reference test or inclusion in a CRS, we performed a systematic review of the typhoid literature to include all index/reference test combinations observed. We described the landscape of comparisons performed, showed results of a meta-analysis on the accuracy of the more common combinations, and evaluated sources of variability based on study quality. This wide-ranging meta-analysis suggests that no single test has sufficiently good performance but some existing diagnostics may be useful as part of a CRS. Additionally, based on findings from the meta-analysis and a constructed numerical example demonstrating the use of CRS, we proposed necessary criteria and potential components of a typhoid CRS to guide future recommendations. Agreement and adoption by all investigators of a standardized CRS is requisite, and would improve comparison of new diagnostics across independent studies, leading to the identification of a better reference test and improved confidence in prevalence estimates.
| S-EPMC4643909 | biostudies-literature
Project description:Missing data are pandemic and a central problem for epidemiology. Missing data reduce precision and can cause notable bias. There remain too few simple published examples detailing types of missing data and illustrating their possible impact on results. Here we take an example randomized trial that was not subject to missing data and induce missing data to illustrate 4 scenarios in which outcomes are 1) missing completely at random, 2) missing at random with positivity, 3) missing at random without positivity, and 4) missing not at random. We demonstrate that accounting for missing data is generally a better strategy than ignoring missing data, which unfortunately remains a standard approach in epidemiology.
| S-EPMC10144620 | biostudies-literature
Project description:ObjectivesThe objective of this study is to introduce methods to use all of the information without omission when individual studies provide multiple effect sizes according to multiple cut-off values (thresholds) during diagnostic test accuracy (DTA) for data integration. For diagnostic test meta-analysis, a general performance method for synthesizing data according to one cut value in one study and a performance method for synthesizing data according to two or more cut values in one study were compared and analyzed.MethodsAs sample data for meta-analysis of DTA studies, 13 DTA studies on prostate cancer (34 effect sizes including total cut-offs) were collected. The summary statistics were calculated and the summary line was analyzed using the "meta", "mada", and "diagmeta" packagesof the R software.ResultsThe summary statistics of the random effect model univariate analysis of the "meta" package with a single cut-off corresponding to the highest Youden index in a single study and those of the bivariate analysis of the "mada" package were highly similar. However, in the bivariate analysis of the "diagmeta" package including all cut-off values, the sensitivity decreased and the specificity increased as the amount of data increased.ConclusionsConsidering the heterogeneity of the summary receiver op erating characteristic curve and the use of all given cut-offs, the use of the bivariate analysis model of the "diagmeta" package is recommended. This study focused on practical methods of DTA rather than theoretical concepts for use by researchers whose fields of study are non-statistics related. By performing this study, we hope that many researchers will use R software to determine the DTA more easily, and that there will be greater interest in related research.
| S-EPMC10106543 | biostudies-literature
Project description:Researchers faced with incomplete data are encouraged to consider whether their data are 'missing completely at random' (MCAR), 'missing at random' (MAR) or 'missing not at random' (MNAR) when planning their analysis. However, there are two major problems with this classification as originally defined by Rubin in the 1970s. First, when there are missing data in multiple variables, the plausibility of the MAR assumption is difficult to assess using substantive knowledge and is more stringent than is generally appreciated. Second, although MCAR and MAR are sufficient conditions for consistent estimation with specific methods, they are not necessary conditions and therefore this categorization does not directly determine the best approach for handling the missing data in an analysis. How best to handle missing data depends on the assumed causal relationships between variables and their missingness, and what these relationships imply in terms of the 'recoverability' of the target estimand (the population parameter that encodes the answer to the underlying research question). Recoverability is defined as whether the estimand can be consistently estimated from the patterns and associations in the observed data without needing to invoke external information on the extent to which the distribution of missing values might differ from that of observed values. In this manuscript we outline an approach for deciding which method to use to handle multivariable missing data in an analysis, using directed acyclic graphs to depict missingness assumptions and determining the implications in terms of recoverability of the target estimand.
| S-EPMC10396404 | biostudies-literature
Project description:BackgroundPoor reporting quality in diagnostic accuracy studies hampers an adequate judgment of the validity of the study. The Standards for Reporting of Diagnostic Accuracy Studies (STARD) statement was published to improve the reporting quality of diagnostic accuracy studies. This study aimed to evaluate the adherence of diagnostic accuracy studies published in Annals of Laboratory Medicine (ALM) to STARD 2015 and to identify directions for improvement in the reporting quality of these studies.MethodsTwo independent authors assessed articles published in ALM between 2012-2018 for compliance with 30 STARD 2015 checklist items to identify all eligible diagnostic accuracy studies published during this period. We included 66 diagnostic accuracy studies. A total of the fulfilled STARD items were calculated, and adherence was analyzed on an individual-item basis.ResultsThe overall mean±SD number of STARD items reported for the included studies was 11.2±2.7. Only five (7.6%) studies adhered to more than 50% of the 30 items. No study satisfied more than 80% of the items. Large variability in adherence to reporting standards was detected across items, ranging from 0% to 100%.ConclusionsAdherence to STARD 2015 is suboptimal among diagnostic accuracy studies published in ALM. Our study emphasizes the necessity of adherence to STARD to improve the reporting quality of future diagnostic accuracy studies to be published in ALM.
| S-EPMC6933069 | biostudies-literature
Project description:Genotype imputation is an important tool in human genetics studies, which uses reference sets with known genotypes and prior knowledge on linkage disequilibrium and recombination rates to infer un-typed alleles for human genetic variations at a low cost. The reference sets used by current imputation approaches are based on HapMap data, and/or based on recently available next-generation sequencing (NGS) data such as data generated by the 1000 Genomes Project. However, with different coverage and call rates for different NGS data sets, how to integrate NGS data sets of different accuracy as well as previously available reference data as references in imputation is not an easy task and has not been systematically investigated. In this study, we performed a comprehensive assessment of three strategies on using NGS data and previously available reference data in genotype imputation for both simulated data and empirical data, in order to obtain guidelines for optimal reference set construction. Briefly, we considered three strategies: strategy 1 uses one NGS data as a reference; strategy 2 imputes samples by using multiple individual data sets of different accuracy as independent references and then combines the imputed samples with samples based on the high accuracy reference selected when overlapping occurs; and strategy 3 combines multiple available data sets as a single reference after imputing each other. We used three software (MACH, IMPUTE2 and BEAGLE) for assessing the performances of these three strategies. Our results show that strategy 2 and strategy 3 have higher imputation accuracy than strategy 1. Particularly, strategy 2 is the best strategy across all the conditions that we have investigated, producing the best accuracy of imputation for rare variant. Our study is helpful in guiding application of imputation methods in next generation association analyses.
| S-EPMC3559437 | biostudies-literature
Project description: Not available
| S-EPMC1373753 | biostudies-literature