Dataset Information

Recalibration Methods for Improved Clinical Utility of Risk Scores.

ABSTRACT:

Background

An established risk model may demonstrate miscalibration, meaning predicted risks do not accurately capture event rates. In some instances, investigators can identify and address the cause of miscalibration. In other circumstances, it may be appropriate to recalibrate the risk model. Existing recalibration methods do not account for settings in which the risk score will be used for risk-based clinical decision making.

Methods

We propose 2 new methods for risk model recalibration when the intended purpose of the risk model is to prescribe an intervention to high-risk individuals. Our measure of risk model clinical utility is standardized net benefit. The first method is a weighted strategy that prioritizes good calibration at or around the critical risk threshold. The second method uses constrained optimization to produce a recalibrated risk model with maximum possible net benefit, thereby prioritizing good calibration around the critical risk threshold. We also propose a graphical tool for assessing the potential for recalibration to improve the net benefit of a risk model. We illustrate these methods by recalibrating the American College of Cardiology (ACC)-American Heart Association (AHA) atherosclerotic cardiovascular disease (ASCVD) risk score within the Multi-Ethnic Study of Atherosclerosis (MESA) cohort.

Results

New methods are implemented in the R package ClinicalUtilityRecal. Recalibrating the ACC-AHA-ASCVD risk score for a MESA subcohort results in higher estimated net benefit using the proposed methods compared with existing methods, with improved calibration in the most clinically impactful regions of risk.

Conclusion

The proposed methods target good calibration for critical risks and can improve the net benefit of a risk model. We recommend constrained optimization when the risk model net benefit is paramount. The weighted approach can be considered when good calibration over an interval of risks is important.

SUBMITTER: Mishra A

PROVIDER: S-EPMC8977399 | biostudies-literature | 2022 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Recalibration Methods for Improved Clinical Utility of Risk Scores.

Mishra Anu A McClelland Robyn L RL Inoue Lurdes Y T LYT Kerr Kathleen F KF

Medical decision making : an international journal of the Society for Medical Decision Making 20211004 4

<h4>Background</h4>An established risk model may demonstrate miscalibration, meaning predicted risks do not accurately capture event rates. In some instances, investigators can identify and address the cause of miscalibration. In other circumstances, it may be appropriate to recalibrate the risk model. Existing recalibration methods do not account for settings in which the risk score will be used for risk-based clinical decision making.<h4>Methods</h4>We propose 2 new methods for risk model reca ...[more]

PMID: 34605718

Similar Datasets

Project description:BackgroundGenome-wide association studies have identified thousands of disease-associated single nucleotide polymorphisms (SNPs). A subset of these SNPs may be additively combined to generate genetic risk scores (GRSs) that confer risk for a specific disease. Although the clinical validity of GRSs to predict risk of specific diseases has been well established, there is still a great need to determine their clinical utility by applying GRSs in primary care for cancer risk assessment and targeted intervention.MethodsThis clinical study involved 281 primary care patients without a personal history of breast, prostate or colorectal cancer who were 40-70 years old. DNA was obtained from a pre-existing biobank at NorthShore University HealthSystem. GRSs for colorectal cancer and breast or prostate cancer were calculated and shared with participants through their primary care provider. Additional data was gathered using questionnaires as well as electronic medical record information. A t-test or Chi-square test was applied for comparison of demographic and key clinical variables among different groups.ResultsThe median age of the 281 participants was 58 years and the majority were female (66.6%). One hundred one (36.9%) participants received 2 low risk scores, 99 (35.2%) received 1 low risk and 1 average risk score, 37 (13.2%) received 1 low risk and 1 high risk score, 23 (8.2%) received 2 average risk scores, 21 (7.5%) received 1 average risk and 1 high risk score, and no one received 2 high risk scores. Before receiving GRSs, younger patients and women reported significantly more worry about risk of developing cancer. After receiving GRSs, those who received at least one high GRS reported significantly more worry about developing cancer. There were no significant differences found between gender, age, or GRS with regards to participants' reported optimism about their future health neither before nor after receiving GRS results.ConclusionsGenetic risk scores that quantify an individual's risk of developing breast, prostate and colorectal cancers as compared with a race-defined population average risk have potential clinical utility as a tool for risk stratification and to guide cancer screening in a primary care setting.

Dataset Information

Recalibration Methods for Improved Clinical Utility of Risk Scores.

Background

Methods

Results

Conclusion

Publications

Recalibration Methods for Improved Clinical Utility of Risk Scores.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets