Dataset Information

Sample size and precision of estimates in studies of depression screening tool accuracy: A meta-research review of studies published in 2018-2021.

ABSTRACT:

Objectives

Depression screening tool accuracy studies should be conducted with large enough sample sizes to generate precise accuracy estimates. We assessed the proportion of recently published depression screening tool diagnostic accuracy studies that reported sample size calculations; the proportion that provided confidence intervals (CIs); and precision, based on the width and lower bounds of 95% CIs for sensitivity and specificity. In addition, we assessed whether these results have improved since a previous review of studies published in 2013-2015.

Methods

MEDLINE was searched from January 1, 2018, through May 21, 2021.

Results

Twelve of 106 primary studies (11%) described a viable sample size calculation, which represented an improvement of 8% since the last review. Thirty-six studies (34%) provided reasonably accurate CIs. Of 103 studies where 95% CIs were provided or could be calculated, seven (7%) had sensitivity CI widths of ≤10%, whereas 58 (56%) had widths of ≥21%. Eighty-four studies (82%) had lower bounds of CIs <80% for sensitivity and 77 studies (75%) for specificity. These results were similar to those reported previously.

Conclusion

Few studies reported sample size calculations, and the number of included individuals in most studies was too small to generate reasonably precise accuracy estimates.

SUBMITTER: Nassar EL

PROVIDER: S-EPMC9159687 | biostudies-literature | 2022 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Sample size and precision of estimates in studies of depression screening tool accuracy: A meta-research review of studies published in 2018-2021.

Nassar Elsa-Lynn EL Levis Brooke B Neyer Marieke A MA Rice Danielle B DB Booij Linda L Benedetti Andrea A Thombs Brett D BD

International journal of methods in psychiatric research 20220401 2

<h4>Objectives</h4>Depression screening tool accuracy studies should be conducted with large enough sample sizes to generate precise accuracy estimates. We assessed the proportion of recently published depression screening tool diagnostic accuracy studies that reported sample size calculations; the proportion that provided confidence intervals (CIs); and precision, based on the width and lower bounds of 95% CIs for sensitivity and specificity. In addition, we assessed whether these results have ...[more]

PMID: 35362161

Similar Datasets

Project description:Background. National valuation studies are costly, with ∼1000 face-to-face interviews recommended, and some countries may deem such studies infeasible. Building on previous studies exploring sample size, we determined the effect of sample size and alternative model specifications on prediction accuracy of modeled coefficients in EQ-5D-5L value set generating regression analyses. Methods. Data sets (n = 50 to ∼1000) were simulated from 3 valuation studies, resampled at the respondent level and randomly drawn 1000 times with replacement. We estimated utilities for each subsample with leave-one-out at the block level using regression models (8 or 20 parameter; with or without a random intercept; time tradeoff [TTO] data only or TTO + discrete choice experiment [DCE] data). Prediction accuracy, root mean square error (RMSE), was calculated by comparing to censored mean predicted values to the left-out block in the full data set. Linear regression was used to estimate the relative effect of changes in sample size and each model specification. Results. Results showed that doubling the sample size decreased RMSE by on average 0.012. Effects of other model specifications were smaller but can when combined compensate for loss in prediction accuracy from a small sample size. For models using TTO data only, 8-parameter models clearly outperformed 20-parameter models. Adding a random intercept, or including DCE responses, also improved mean RMSE, most prominently for variants of the 20-parameter models. Conclusions. The prediction accuracy impact of further increases in sample size after 300 to 500 were smaller than the impact of combining alternative modeling choices. Hybrid modeling, use of constrained models, and inclusion of random intercepts all substantially improve the expected prediction accuracy. Beyond a minimum of 300 to 500 respondents, the sample size may be better informed by other considerations, such as legitimacy and representativeness, than by the technical prediction accuracy achievable.HighlightsIncreases in sample size beyond a minimum in the range of 300 to 500 respondents provide smaller gains in expected prediction accuracy than alternative modeling approaches.Constrained, nonlinear models; time tradeoff + discrete choice experiment hybrid modeling; and including a random intercept all improved the prediction accuracy of models estimating values for the EQ-5D-5L based on data from 3 different valuation studies.The tested modeling choices can compensate for smaller sample sizes.

Project description:Applicability of intravoxel incoherent motion (IVIM) imaging in the clinical setting is hampered by the limited reliability in particular of the perfusion-related parameter estimates. To alleviate this problem, various advanced postprocessing methods have been introduced. However, the underlying algorithms are not readily available and generally suffer from an increased computational burden. Contrary, several computationally fast image denoising methods have recently been proposed which are accessible online and may improve reliability of IVIM parameter estimates. The objective of the present work is to investigate the impact of image denoising on accuracy and precision of IVIM parameter estimates using comprehensive in-silico and in-vivo experiments. Image denoising is performed with four different algorithms that work on magnitude data: two algorithms which are based on nonlocal means (NLM) filtering, one algorithm that relies on local principal component analysis (LPCA) of the diffusion-weighted images, and another algorithms that exploits joint rank and edge constraints (JREC). Accuracy and precision of IVIM parameter estimates is investigated in an in-silico brain phantom and an in-vivo ground truth as a function of the signal-to-noise ratio for spatially homogenous and inhomogenous levels of Rician noise. Moreover, precision is evaluated using bootstrap analysis of in-vivo measurements. In the experiments, IVIM parameters are computed a) by using a segmented fit method and b) by performing a biexponential fit of the entire attenuation curve based on nonlinear least squares estimates. Irrespective of the fit method, the results demonstrate that reliability of IVIM parameter estimates is substantially improved by image denoising. The experiments show that the LPCA and the JREC algorithms perform in a similar manner and outperform the NLM-related methods. Relative to noisy data, accuracy of the IVIM parameters in the in-silico phantom improves after image denoising by 76-79%, 79-81%, 84-99% and precision by 74-80%, 80-83%, 84-95% for the perfusion fraction, the diffusion coefficient, and the pseudodiffusion coefficient, respectively, when the segmented fit method is used. Beyond that, the simulations reveal that denoising performance is not impeded by spatially inhomogeneous levels of Rician noise in the image. Since all investigated algorithms are freely available and work on magnitude data they can be readily applied in the clinical setting which may foster transition of IVIM imaging into clinical practice.

Dataset Information

Sample size and precision of estimates in studies of depression screening tool accuracy: A meta-research review of studies published in 2018-2021.

Objectives

Methods

Results

Conclusion

Publications

Sample size and precision of estimates in studies of depression screening tool accuracy: A meta-research review of studies published in 2018-2021.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets