Dataset Information

Leveraging Decision Curve Analysis to Improve Clinical Application of Surgical Risk Calculators.

ABSTRACT:

Background

Surgical risk calculators (SRCs) have been developed for estimation of postoperative complications but do not directly inform decision-making. Decision curve analysis (DCA) is a method for evaluating prediction models, measuring their utility in guiding decisions. We aimed to analyze the utility of SRCs to guide both preoperative and postoperative management of patients undergoing hepatopancreaticobiliary surgery by using DCA.

Methods

A single-institution, retrospective review of patients undergoing hepatopancreaticobiliary operations between 2015 and 2017 was performed. Estimation of postoperative complications was conducted using the American College of Surgeons SRC [ACS-SRC] and the Predictive OpTimal Trees in Emergency Surgery Risk (POTTER) calculator; risks were compared with observed outcomes. DCA was used to model optimal patient selection for risk prevention strategies and to compare the relative performance of the ACS-SRC and POTTER calculators.

Results

A total of 994 patients were included in the analysis. C-statistics for the ACS-SRC prediction of 12 postoperative complications ranged from 0.546 to 0.782. DCA revealed that an ACS-SRC-guided readmission prevention intervention, when compared with an all-or-none approach, yielded a superior net benefit for patients with estimated risk between 5% and 20%. Comparison of SRCs for venous thromboembolism intervention demonstrated superiority of the ACS-SRC for thresholds for intervention between 2% and 4% with the POTTER calculator performing superiorly between 4% and 8% estimated risk.

Conclusions

SRCs can be used not only to predict complication risk but also to guide risk prevention strategies. This methodology should be incorporated into external validations of future risk calculators and can be applied for institution-specific quality improvement initiatives to improve patient outcomes.

SUBMITTER: Dadashzadeh ER

PROVIDER: S-EPMC8991373 | biostudies-literature | 2021 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Leveraging Decision Curve Analysis to Improve Clinical Application of Surgical Risk Calculators.

Dadashzadeh Esmaeel Reza ER Bou-Samra Patrick P Huckaby Lauren V LV Nebbia Giacomo G Handzel Robert M RM Varley Patrick R PR Wu Shandong S Tsung Allan A

The Journal of surgical research 20210105

<h4>Background</h4>Surgical risk calculators (SRCs) have been developed for estimation of postoperative complications but do not directly inform decision-making. Decision curve analysis (DCA) is a method for evaluating prediction models, measuring their utility in guiding decisions. We aimed to analyze the utility of SRCs to guide both preoperative and postoperative management of patients undergoing hepatopancreaticobiliary surgery by using DCA.<h4>Methods</h4>A single-institution, retrospective ...[more]

PMID: 33418322

Similar Datasets

Project description:BackgroundClinical scores are frequently used in the diagnosis and management of stroke. While medical calculators are increasingly important support tools for clinical decisions, the uptake and use of common medical calculators for stroke remain poorly characterized.ObjectiveWe aimed to describe use patterns in frequently used stroke-related medical calculators for clinical decisions from a web-based support system.MethodsWe conducted a retrospective study of calculators from MDCalc, a web-based and mobile app-based medical calculator platform based in the United States. We analyzed metadata tags from MDCalc's calculator use data to identify all calculators related to stroke. Using relative page views as a measure of calculator use, we determined the 5 most frequently used stroke-related calculators between January 2016 and December 2018. For all 5 calculators, we determined cumulative and quarterly use, mode of access (eg, app or web browser), and both US and international distributions of use. We compared cumulative use in the 2016-2018 period with use from January 2011 to December 2015.ResultsOver the study period, we identified 454 MDCalc calculators, of which 48 (10.6%) were related to stroke. Of these, the 5 most frequently used calculators were the CHA2DS2-VASc score for atrial fibrillation stroke risk calculator (5.5% of total and 32% of stroke-related page views), the Mean Arterial Pressure calculator (2.4% of total and 14.0% of stroke-related page views), the HAS-BLED score for major bleeding risk (1.9% of total and 11.4% of stroke-related page views), the National Institutes of Health Stroke Scale (NIHSS) score calculator (1.7% of total and 10.1% of stroke-related page views), and the CHADS2 score for atrial fibrillation stroke risk calculator (1.4% of total and 8.1% of stroke-related page views). Web browser was the most common mode of access, accounting for 82.7%-91.2% of individual stroke calculator page views. Access originated most frequently from the most populated regions within the United States. Internationally, use originated mostly from English-language countries. The NIHSS score calculator demonstrated the greatest increase in page views (238.1% increase) between the first and last quarters of the study period.ConclusionsThe most frequently used stroke calculators were the CHA2DS2-VASc, Mean Arterial Pressure, HAS-BLED, NIHSS, and CHADS2. These were mainly accessed by web browser, from English-speaking countries, and from highly populated areas. Further studies should investigate barriers to stroke calculator adoption and the effect of calculator use on the application of best practices in cerebrovascular disease.

Project description:IntroductionClinical prediction and decision tools that generate outcome-based risk stratification and/or intervention recommendations are prevalent. Appropriate use and validity of these tools, especially those that inform complex clinical decisions, remains unclear. The objective of this study was to assess the methodologic quality and applicability of clinical risk scoring tools used to guide hospitalization decision-making.MethodsIn February 2021, a comprehensive search was performed of a clinical calculator online database (mdcalc.com) that is publicly available and well-known to clinicians. The primary reference for any calculator tool informing outpatient versus inpatient disposition was considered for inclusion. Studies were restricted to the adult, acute care population. Those focused on obstetrics/gynecology or critical care admission were excluded. The Wasson-Laupacis framework of methodologic standards for clinical prediction rules was applied to each study.ResultsA total of 22 calculators provided hospital admission recommendations for 9 discrete medical conditions using adverse events (14/22), mortality (6/22), or confirmatory diagnosis (2/22) as outcomes of interest. The most commonly met methodologic standards included mathematical technique description (22/22) and clinical sensibility (22/22) and least commonly met included reproducibility of the rule (1/22) and measurement of effect on clinical use (1/22). Description of the studied population was often lacking, especially patient race/ethnicity (2/22) and mental or behavioral health (0/22). Only one study reported any item related to social determinants of health.ConclusionStudies commonly do not meet rigorous methodologic standards and often fail to report pertinent details that would guide applicability. These clinical tools focus primarily on specific disease entities and clinical variables, missing the breadth of information necessary to make a disposition determination and raise significant validation and generalizability concerns.

Project description:BackgroundNon-alcoholic fatty liver disease (NAFLD) is the commonest liver condition in the western world and is directly linked to obesity and the metabolic syndrome. Elevated body mass index is regarded as a major risk factor of NAFL (steatosis) and NAFLD fibrosis. Using data from the Avon Longitudinal Study of Parents and Children (ALSPAC), we sought to investigate whether other variables from adolescence could improve prediction of future NAFL and NAFLD fibrosis risk at 24 years, above BMI and sex.MethodsAged 24 years, 4018 ALSPAC participants had transient elastography (TE) and controlled attenuation parameter (CAP) measurement using Echosens 502 Touch. 513 participants with harmful alcohol consumption were excluded. Logistic regression models examined which variables measured at 17 years were predictive of NAFL and NAFLD fibrosis in young adults. Predictors included sex, BMI, central adiposity, lipid profile, blood pressure, liver function tests, homeostatic model assessment for insulin resistance (HOMA-IR), and ultrasound defined NAFL at 17 years (when examining fibrosis outcomes). A model including all these variables was termed "routine clinical measures". Models were compared using area under the receiver operator curve (AUROC) and Bayesian Information Criterion (BIC), analysis, which penalises model complexity. Models were tested in all participants and those with overweight or obese standardised BMIs (BMI SDS) centiles at the 17-year time point. A decision curve analysis (DCA) was performed to evaluate the clinical utility of models in overweight and obese adolescents predicting NAFLD fibrosis at a threshold probability of 0.1.ResultsThe "routine clinical measures" model had the highest AUROC for predicting NAFL in all adolescent participants (AUROC 0.79 [SD 0.00]) and those with an overweight/obese BMI SDS centile (AUROC 0.77 [SD 0.01]). According to BIC analysis, insulin resistance was the best predictor of NAFL in all adolescents, whilst central adiposity was the best predictor in those with an overweight/obese BMI SDS centile. The "routine clinical measures" model also had the highest AUROC for predicting NAFLD fibrosis in all adolescent participants (AUROC 0.78 [SD 0.02]) and participants with an overweight/obese BMI SDS centile (AUROC 0.84 [SD 0.03]). However, following BIC analysis, BMI was the best predictor of NAFLD fibrosis in all adolescents including those with an overweight/obese BMI SDS centile. A decision curve analysis examining overweight/obese adolescent participants showed the model that had the greatest net benefit for increased NAFLD fibrosis detection, above a treat all overweight and obese adolescents' assumption, was the "routine clinical measures" model. However, the net benefit was marginal (0.0054 [0.0034-0.0075]).ConclusionIn adolescents, routine clinical measures were not superior to central adiposity and BMI at predicting NAFL and NAFLD fibrosis respectively in young adulthood. Additional routine clinical measurements do provide incremental benefit in detecting true positive fibrosis cases, but the benefit is small. Thus, to reduce morbidity and mortality associated with NASH cirrhosis in adults, the ultimate end point of NAFLD, the focus must be on obesity management at a population level.

Project description:BackgroundElectronic medical records are widely used in family practices across Canada and can improve health outcomes. However, recent reports indicate that physicians using electronic medical records work longer and have less direct patient contact which may contribute to burnout. Therefore, new and innovative digital tools are essential to reduce physician workloads and improve patient-physician interaction to address physician burnout. The objective of this study was to assess the efficiency and accuracy of clinical decision-making when using a new preventive care point-of-care clinical decision support system (CDSS). An estimate of the potential annual time savings was also determined. This study also assessed physician reported perceived usefulness and ease of use of the CDSS.MethodsQuantitative and qualitative data were collected during this study. Each participant evaluated two simulated patient charts and identified which preventive care metrics were due. The participants recorded their decisions and the time required to assess each chart. Participants then completed a Technology Acceptance Model survey regarding the perceived usefulness and ease of use of the CDSS, which included qualitative feedback. The amount of time saved was determined and participants' clinical decision-making accuracy was scored against current Canadian preventive care guidelines. The number of preventive care specific visits completed per year was determined using clinic billing data.ResultsThe preventive care CDSS saved an average of 195.7 s of chart review time (249.5 s vs 445.2 s; P < 0.001). A total of 1520 preventive visits were performed at Primrose and Bruyère Family Medicine Centres. Extrapolated across the organization, implementation of the new tool could save 82.6 h per year. Decision-making accuracy was not affected by the new tool (78.4% vs 80.9%, P > 0.05). Participants rated the perceived ease of use and usefulness to be very high.ConclusionsNew digital tools may reduce providers' workload without impacting clinical decision-making accuracy. Participants indicated that the preventive care CDSS was useful and easy to use. Further software development and clinical studies are required to further improve and characterize the effect this new CDSS has when implemented in clinical practice.

Project description:BackgroundRisk prediction models have been proposed for various diseases and are being improved as new predictors are identified. A major challenge is to determine whether the newly discovered predictors improve risk prediction. Decision curve analysis has been proposed as an alternative to the area under the curve and net reclassification index to evaluate the performance of prediction models in clinical scenarios. The decision curve computed using the net benefit can evaluate the predictive performance of risk models at a given or range of threshold probabilities. However, when the decision curves for 2 competing models cross in the range of interest, it is difficult to identify the best model as there is no readily available summary measure for evaluating the predictive performance. The key deterrent for using simple measures such as the area under the net benefit curve is the assumption that the threshold probabilities are uniformly distributed among patients.MethodsWe propose a novel measure for performing decision curve analysis. The approach estimates the distribution of threshold probabilities without the need of additional data. Using the estimated distribution of threshold probabilities, the weighted area under the net benefit curve serves as the summary measure to compare risk prediction models in a range of interest.ResultsWe compared 3 different approaches, the standard method, the area under the net benefit curve, and the weighted area under the net benefit curve. Type 1 error and power comparisons demonstrate that the weighted area under the net benefit curve has higher power compared to the other methods. Several simulation studies are presented to demonstrate the improvement in model comparison using the weighted area under the net benefit curve compared to the standard method.ConclusionsThe proposed measure improves decision curve analysis by using the weighted area under the curve and thereby improves the power of the decision curve analysis to compare risk prediction models in a clinical scenario.

Dataset Information

Leveraging Decision Curve Analysis to Improve Clinical Application of Surgical Risk Calculators.

Background

Methods

Results

Conclusions

Publications

Leveraging Decision Curve Analysis to Improve Clinical Application of Surgical Risk Calculators.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets