Dataset Information

Quantifying the Prognostic Value of Preoperative Surgeon Intuition: Comparing Surgeon Intuition and Clinical Risk Prediction as Derived from the American College of Surgeons NSQIP Risk Calculator.

ABSTRACT:

Background

Surgical risk prediction models traditionally use patient attributes and measures of physiology to generate predictions about postoperative outcomes. However, the surgeon's assessment of the patient may be a valuable predictor, given the surgeon's ability to detect and incorporate factors that existing models cannot capture. We compare the predictive utility of surgeon intuition and a risk calculator derived from the American College of Surgeons (ACS) NSQIP.

Study design

From January 10, 2021 to January 9, 2022, surgeons were surveyed immediately before performing surgery to assess their perception of a patient's risk of developing any postoperative complication. Clinical data were abstracted from ACS NSQIP. Both sources of data were independently used to build models to predict the likelihood of a patient experiencing any 30-day postoperative complication as defined by ACS NSQIP.

Results

Preoperative surgeon assessment was obtained for 216 patients. NSQIP data were available for 9,182 patients who underwent general surgery (January 1, 2017 to January 9, 2022). A binomial regression model trained on clinical data alone had an area under the receiver operating characteristic curve (AUC) of 0.83 (95% CI 0.80 to 0.85) in predicting any complication. A model trained on only preoperative surgeon intuition had an AUC of 0.70 (95% CI 0.63 to 0.78). A model trained on surgeon intuition and a subset of clinical predictors had an AUC of 0.83 (95% CI 0.77 to 0.89).

Conclusions

Preoperative surgeon intuition alone is an independent predictor of patient outcomes; however, a risk calculator derived from ACS NSQIP is a more robust predictor of postoperative complication. Combining intuition and clinical data did not strengthen prediction.

SUBMITTER: Marwaha JS

PROVIDER: S-EPMC10192014 | biostudies-literature | 2023 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Quantifying the Prognostic Value of Preoperative Surgeon Intuition: Comparing Surgeon Intuition and Clinical Risk Prediction as Derived from the American College of Surgeons NSQIP Risk Calculator.

Marwaha Jayson S JS Beaulieu-Jones Brendin R BR Berrigan Margaret M Yuan William W Odom Stephen R SR Cook Charles H CH Scott Benjamin B BB Gupta Alok A Parsons Charles S CS Seshadri Anupamaa J AJ Brat Gabriel A GA

Journal of the American College of Surgeons 20230223 6

<h4>Background</h4>Surgical risk prediction models traditionally use patient attributes and measures of physiology to generate predictions about postoperative outcomes. However, the surgeon's assessment of the patient may be a valuable predictor, given the surgeon's ability to detect and incorporate factors that existing models cannot capture. We compare the predictive utility of surgeon intuition and a risk calculator derived from the American College of Surgeons (ACS) NSQIP.<h4>Study design</h ...[more]

PMID: 36815715

Similar Datasets

Project description:BackgroundSurgical wound classification has been used in risk-adjustment models. However, it can be subjective and could potentially improperly bias hospital quality comparisons. The objective is to examine the effect of wound classification on hospital performance risk-adjustment models.Study designRetrospective review of the 2011 American College of Surgeons NSQIP database was conducted for the following wound classification categories: clean, clean-contaminated, contaminated, and dirty-infected. To assess the influence of wound classification on risk adjustment, 2 models were developed for all outcomes: 1 including and 1 excluding wound classification. For each model, hospital postoperative complications were estimated using hierarchical multivariable regression methods. Absolute changes in hospital rank, correlations of odds ratios, and outlier status agreement between models were examined.ResultsOf the 442,149 cases performed in 315 hospitals: 53.6% were classified as clean; 34.2% as clean-contaminated; 6.7% as contaminated; and 5.5% as dirty-infected. The surgical site infection rate was highest in dirty-infected (8.5%) and lowest in clean (1.8%) cases. For overall surgical site infection, the absolute change in risk-adjusted hospital performance rank between models, including vs excluding wound classification, was minimal (mean 4.5 of 315 positions). The correlations between odds ratios of the 2 performance models were nearly perfect (R = 0.9976, p < 0.0001), and outlier status agreement was excellent (κ = 0.95ss08, p < 0.0001). Similar findings were observed in models of subgroups of surgical site infections and other postoperative outcomes.ConclusionsIn circumstances where alternate information is available for risk adjustment, there appear to be minimal differences in performance models that include vs exclude wound classification. Therefore, the American College of Surgeons NSQIP is critically evaluating the continued use of wound classification in hospital performance risk-adjustment models.

Project description:BackgroundAdvances in medicine and surgical techniques make it possible to operate on selected comorbid elderly patients for whom risk assessment is essential. We aimed to validate the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) surgical risk calculator specifically for thoracic surgery.MethodsThis study retrospectively included 283 consecutive patients who all underwent various thoracic surgeries at our center. Considering "serious complication" as the most important outcome, we compared the predicted risk scores with the observed incidence of 30-day morbidity and mortality. We calculated the area under the receiver operating characteristic curve (AUROC) with 95% confidence intervals for each outcome and utilized the Brier score to check the calibration and complication odds ratios above vs. below average risk in all score outcomes with the number of occurred events.ResultsIn our study population, most patients were <65 years old (48%), predominantly male (63%), and overweight or obese (48%). In addition, 13% had severe chronic obstructive pulmonary disease (COPD), and 75% were categorized as American Society of Anesthesiologists (ASA) class III or higher. For "serious complication", AUROC was 59%, and events were equal in patients with above or below average risk scores (P=0.96). AUROC was 67% for "any complication" and 58% for "return to OR", expressing no useful predictive value. The Brier score and odds ratios were low for all outcomes. Dyspnea, ASA class, COPD, and body mass index as single postoperative risk predictors significantly improved the basic model consisting of the logit of the risk calculator alone. Thus, the calculator alone did not perform as well as these single variables did.ConclusionsThe ACS NSQIP surgical risk calculator exhibited low sensitivity, specificity, and low AUROC for postoperative 30-day morbidity and mortality in our study cohort. Therefore, we think it cannot be considered as valid risk estimation tool for general thoracic surgery.

Project description:BackgroundUtilization of large-volume clinical registries for observational research has gained popularity in orthopaedic literature. However, concerns exist regarding inadequate reporting of methodology in this type of research. Despite these concerns, the reproducibility of such studies has not been adequately assessed in existing literature. This study aims to assess the reproducibility of American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) arthroplasty studies on smoking as a risk factor for poor surgical outcomes by employing identical datasets and statistical methods.MethodsA systematic PubMed search between 2013 and 2023 identified ACS-NSQIP studies involving hip or knee arthroplasty and smoking as a potential risk factor for poor surgical outcomes. Each study's methods were reproduced by a trained statistician based on the reported methodology. In cases where certain steps were not explicitly stated, the statistician made informed decisions to reproduce those steps. Adjusted odds ratios (aORs) and p-values (α = 0.05) were compared between the original and reanalyzed datasets.ResultsThe initial search yielded 43 studies, with 11 meeting inclusion criteria resulting in the reanalysis of 268 aORs. Upon reanalysis, 12.69% of the original studies' aORs changed in interpretation, while 13.43% experienced a change in statistical significance. The average magnitude change of each aOR across all studies was 17.22%, and the sample size (N) in reanalysis varied by up to 47.84%. Among the 11 commonly cited studies, approximately one in eight objective conclusions changed in interpretation or statistical significance.ConclusionInconsistent reproducibility exists across many arthroplasty studies that utilize the ACS-NSQIP database. These findings highlight the importance of rigorous reporting of study methodology, data collection, and statistical analyses when utilizing large-volume databases in orthopaedic research. This burden of responsibility should be shared among authors, peer reviewers, and orthopaedic journals to confirm the accuracy and validity of published database research.Level of evidenceThis study systematically reviewed and analyzed, in attempt to reproduce, published arthroplasty studies utilizing ACS-NSQIP database to assess smoking as a potential risk factor for poor surgical outcomes. All analyzed studies included Level III Evidence, therefore this current study compares reproduced Level III Evidence to the original Level III Evidence.

Project description:The aim of this study was to evaluate the benefit of adding hypoalbuminemia to the American College of Surgeons-National Surgical Quality Improvement Program (ACS-NSQIP) Surgical Risk Calculator when predicting postoperative outcomes in colorectal cancer patients.The ACS-NSQIP Surgical Risk Calculator offers qualified risk evaluation in surgical decision-making and informed patient consent. To date, malnutrition defined as hypoalbuminemia, an important independent surgical risk factor in colorectal cancer, is not included.This is a retrospective, multi-institutional study of ACS-NSQIP patients (n = 18,532) who received colorectal surgery from 2009 to 2012. Models were constructed for predicting postoperative mortality and morbidity using the risk factors of the ACS-NSQIP Surgical Risk Calculator before and after adding hypoalbuminemia as a risk factor. The 2 models' performance was then compared using c-statistics and Brier scores. The ACS-NSQIP database in 2008 was used for validation of the created models.The prevalence of hypoalbuminemia (27.8%) is higher in colorectal cancer, when compared with other most common cancers. In univariate analyses, hypoalbuminemia was significantly associated with postoperative mortality and morbidity in colorectal cancer patients. In multivariate logistic regression analyses, 15 postoperative complications, including mortality and serious morbidities, were significantly predicted by hypoalbuminemia. Most of the models with hypoalbuminemia showed better performance and validation in predicting postoperative complications than those without hypoalbuminemia.In colorectal cancer, hypoalbuminemia, with levels below 3.5 g/dL, serves as an excellent assessment tool and preoperative predictor of postoperative outcomes. When combined with hypoalbuminemia as a risk factor, the ACS-NSQIP Surgical Risk Calculator offers more accurate information and estimation of surgical risks to patients and surgeons when choosing treatment options.

Dataset Information

Quantifying the Prognostic Value of Preoperative Surgeon Intuition: Comparing Surgeon Intuition and Clinical Risk Prediction as Derived from the American College of Surgeons NSQIP Risk Calculator.

Background

Study design

Results

Conclusions

Publications

Quantifying the Prognostic Value of Preoperative Surgeon Intuition: Comparing Surgeon Intuition and Clinical Risk Prediction as Derived from the American College of Surgeons NSQIP Risk Calculator.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets