Equalization of four cardiovascular risk algorithms after systematic recalibration: individual-participant meta-analysis of 86 prospective studies.
ABSTRACT: AIMS:There is debate about the optimum algorithm for cardiovascular disease (CVD) risk estimation. We conducted head-to-head comparisons of four algorithms recommended by primary prevention guidelines, before and after 'recalibration', a method that adapts risk algorithms to take account of differences in the risk characteristics of the populations being studied. METHODS AND RESULTS:Using individual-participant data on 360?737 participants without CVD at baseline in 86 prospective studies from 22 countries, we compared the Framingham risk score (FRS), Systematic COronary Risk Evaluation (SCORE), pooled cohort equations (PCE), and Reynolds risk score (RRS). We calculated measures of risk discrimination and calibration, and modelled clinical implications of initiating statin therapy in people judged to be at 'high' 10 year CVD risk. Original risk algorithms were recalibrated using the risk factor profile and CVD incidence of target populations. The four algorithms had similar risk discrimination. Before recalibration, FRS, SCORE, and PCE over-predicted CVD risk on average by 10%, 52%, and 41%, respectively, whereas RRS under-predicted by 10%. Original versions of algorithms classified 29-39% of individuals aged ?40?years as high risk. By contrast, recalibration reduced this proportion to 22-24% for every algorithm. We estimated that to prevent one CVD event, it would be necessary to initiate statin therapy in 44-51 such individuals using original algorithms, in contrast to 37-39 individuals with recalibrated algorithms. CONCLUSION:Before recalibration, the clinical performance of four widely used CVD risk algorithms varied substantially. By contrast, simple recalibration nearly equalized their performance and improved modelled targeting of preventive action to clinical need.
Project description:This research aims to assess application of different cardiovascular disease (CVD) mortality risk prediction models in Chinese rural population. Data was collected from a 6-year follow-up survey in rural area of Henan Province, China. 10338 participants aged 40 to 65 years were included. Baseline study was conducted between 2007 and 2008, and followed up from 2013 to 2014. Seven models: general Framingham risk score (general-FRS), simplified-FRS, Systematic Coronary Risk Evaluation for high (SCORE-high), SCORE-low, Chinese ischemic CVD (CN-ICVD), Pooled Cohort Risk Equation for white (PCE-white) and for African-American (PCE-AA) were assessed and recalibrated. The model performance was evaluated by C-statistics and modified Nam-D'Agostino test. 168 CVD deaths occurred during follow-up. All seven models showed moderate C-statics ranging from 0.727 to 0.744. Following recalibration, general-FRS, simplified-FRS, CN-ICVD, PCE-white and PCE-AA had improved C-statistics of 0.776, 0.795, 0.793, 0.779, and 0.776 for men and 0.756, 0.753, 0.755, 0.758 and 0.760 for women, respectively. Calibrations ?2 of general-FRS, simplified-FRS, SCORE-high, CN-ICVD and PCE-AA model for men, and general-FRS, CN-ICVD and PCE-white model for women were statistically acceptable, indicating these models predicts CVD mortality risk more accurately than others and could be recommended in Chinese rural population.
Project description:BACKGROUND:To investigate the performance of the 2013 American College of Cardiology/American Heart Association Pooled Cohort Equations (PCE) in a large, prospective, community-based cohort in Korea and to compare it with that of the Framingham Global Cardiovascular Disease Risk Score (FRS-CVD) and the Korean Risk Prediction Model (KRPM). METHODS:In the Korean Genome and Epidemiology Study (KOGES)-Ansan and Ansung study, we evaluated calibration and discrimination of the PCE for non-Hispanic whites (PCE-WH) and for African Americans (PCE-AA) and compared their predictive abilities with the FRS-CVD and the KRPM. RESULTS:The present study included 7,932 individuals (3,778 men and 4,154 women). The PCE-WH and PCE-AA moderately overestimated the risk of atherosclerotic cardiovascular disease (ASCVD) for men (6% and 13%, respectively) but underestimated the risk for women (-49% and -25%, respectively). The FRS-CVD overestimated ASCVD risk for men (91%) but provided a good risk prediction for women (3%). The KRPM underestimated ASCVD risk for men (-31%) and women (-31%). All the risk prediction models showed good discrimination in both men (C-statistic 0.730 to 0.735) and women (C-statistic 0.726 to 0.732). Recalibration of the PCE using data from the KOGES-Ansan and Ansung study substantially improved the predictive accuracy in men. CONCLUSION:In the KOGES-Ansan and Ansung study, the PCE overestimated ASCVD risk for men and underestimated the risk for women. The PCE-WH and the FRS-CVD provided an accurate prediction of ASCVD in men and women, respectively.
Project description:Background To externally validate the Prediction for ASCVD Risk in China (PAR) risk equation for predicting the 5-year atherosclerotic cardiovascular disease (ASCVD) risk in the Uyghur and Kazakh populations from rural areas in northwestern China and compare its performance with those of the pooled cohort equations (PCE) and Framingham risk score (FRS). Methods The final analysis included 3347 subjects aged 40–74?years without CVD at baseline. The 5-year ASCVD risk was calculated using the PAR, PCE, and FRS. Discrimination, calibration, and clinical usefulness of the three equations in predicting the 5-year ASCVD risk were assessed before and after recalibration. Results Of 3347 included subjects, 1839 were female. We observed 286 ASCVD events in within 5-year follow-up. All three risk equations had moderate discrimination in both men and women. C-indices of PAR, PCE, and FRS were 0.727 (95% CI, 0.725–0.729), 0.727 (95% CI, 0.725–0.729), and 0.740 (95% CI, 0.738–0.742), respectively, in men; the corresponding C-indices were 0.738 (95% CI, 0.737–0.739), 0.731 (95% CI, 0.730–0.732), and 0.761 (95% CI, 0.760–0.762), respectively, in women. PCE, PAR and FRS substantially underestimated the 5-year ASCVD risk in women by 70, 23 and 51%, respectively. However, PAR and FRS fairly predicted the risk in men and PAR was well calibrated. The calibrations of the three risk equations could be changed by recalibration. The decision curve analyses demonstrated that at the threshold risk of 5%, PCE was the most clinically useful in both men and women after recalibration. Conclusions All three risk equations underestimated the 5-year ASCVD risk in women, while PAR and FRS fairly predicted that in men. However, the results of predictive performances for three risk equations are inconsistent, more accurate risk equations are required in the primary prevention of ASCVD aiming to this Uyghur and Kazakh populations.
Project description:Background:Chronic kidney disease (CKD) measures (estimated glomerular filtration rate [eGFR] and albuminuria) are frequently assessed in clinical practice and improve the prediction of incident cardiovascular disease (CVD), yet most major clinical guidelines do not have a standardized approach for incorporating these measures into CVD risk prediction. "CKD Patch" is a validated method to calibrate and improve the predicted risk from established equations according to CKD measures. Methods:Utilizing data from 4,143,535 adults from 35 datasets, we developed several "CKD Patches" incorporating eGFR and albuminuria, to enhance prediction of risk of atherosclerotic CVD (ASCVD) by the Pooled Cohort Equation (PCE) and CVD mortality by Systematic COronary Risk Evaluation (SCORE). The risk enhancement by CKD Patch was determined by the deviation between individual CKD measures and the values expected from their traditional CVD risk factors and the hazard ratios for eGFR and albuminuria. We then validated this approach among 4,932,824 adults from 37 independent datasets, comparing the original PCE and SCORE equations (recalibrated in each dataset) to those with addition of CKD Patch. Findings:We confirmed the prediction improvement with the CKD Patch for CVD mortality beyond SCORE and ASCVD beyond PCE in validation datasets (?c-statistic 0.027 [95% CI 0.018-0.036] and 0.010 [0.007-0.013] and categorical net reclassification improvement 0.080 [0.032-0.127] and 0.056 [0.044-0.067], respectively). The median (IQI) of the ratio of predicted risk for CVD mortality with CKD Patch vs. the original prediction with SCORE was 2.64 (1.89-3.40) in very high-risk CKD (e.g., eGFR 30-44?ml/min/1.73m2 with albuminuria ?30?mg/g), 1.86 (1.48-2.44) in high-risk CKD (e.g., eGFR 45-59?ml/min/1.73m2 with albuminuria 30-299?mg/g), and 1.37 (1.14-1.69) in moderate risk CKD (e.g., eGFR 60-89?ml/min/1.73m2 with albuminuria 30-299?mg/g), indicating considerable risk underestimation in CKD with SCORE. The corresponding estimates for ASCVD with PCE were 1.55 (1.37-1.81), 1.24 (1.10-1.54), and 1.21 (0.98-1.46). Interpretation:The "CKD Patch" can be used to quantitatively enhance ASCVD and CVD mortality risk prediction equations recommended in major US and European guidelines according to CKD measures, when available. Funding:US National Kidney Foundation and the NIDDK.
Project description:BACKGROUND:Cardiovascular disease (CVD) risk prediction models are often used to identify individuals at high risk of CVD events. Providing preventive treatment to these individuals may then reduce the CVD burden at population level. However, different prediction models may predict different (sets of) CVD outcomes which may lead to variation in selection of high risk individuals. Here, it is investigated if the use of different prediction models may actually lead to different treatment recommendations in clinical practice. METHOD:The exact definition of and the event types included in the predicted outcomes of four widely used CVD risk prediction models (ATP-III, Framingham (FRS), Pooled Cohort Equations (PCE) and SCORE) was determined according to ICD-10 codes. The models were applied to a Dutch population cohort (n = 18,137) to predict the 10-year CVD risks. Finally, treatment recommendations, based on predicted risks and the treatment threshold associated with each model, were investigated and compared across models. RESULTS:Due to the different definitions of predicted outcomes, the predicted risks varied widely, with an average 10-year CVD risk of 1.2% (ATP), 5.2% (FRS), 1.9% (PCE), and 0.7% (SCORE). Given the variation in predicted risks and recommended treatment thresholds, preventive drugs would be prescribed for 0.2%, 14.9%, 4.4%, and 2.0% of all individuals when using ATP, FRS, PCE and SCORE, respectively. CONCLUSION:Widely used CVD prediction models vary substantially regarding their outcomes and associated absolute risk estimates. Consequently, absolute predicted 10-year risks from different prediction models cannot be compared directly. Furthermore, treatment decisions often depend on which prediction model is applied and its recommended risk threshold, introducing unwanted practice variation into risk-based preventive strategies for CVD.
Project description:Accurately estimating cardiovascular risk is fundamental to good decision-making in cardiovascular disease (CVD) prevention, but risk scores developed in one population often perform poorly in dissimilar populations. We sought to examine whether a large integrated health system can use their electronic health data to better predict individual patients' risk of developing CVD.We created a cohort using all patients ages 45-80 who used Department of Veterans Affairs (VA) ambulatory care services in 2006 with no history of CVD, heart failure, or loop diuretics. Our outcome variable was new-onset CVD in 2007-2011. We then developed a series of recalibrated scores, including a fully refit "VA Risk Score-CVD (VARS-CVD)." We tested the different scores using standard measures of prediction quality.For the 1,512,092 patients in the study, the Atherosclerotic cardiovascular disease risk score had similar discrimination as the VARS-CVD (c-statistic of 0.66 in men and 0.73 in women), but the Atherosclerotic cardiovascular disease model had poor calibration, predicting 63% more events than observed. Calibration was excellent in the fully recalibrated VARS-CVD tool, but simpler techniques tested proved less reliable.We found that local electronic health record data can be used to estimate CVD better than an established risk score based on research populations. Recalibration improved estimates dramatically, and the type of recalibration was important. Such tools can also easily be integrated into health system's electronic health record and can be more readily updated.
Project description:Persisting high levels of cardiovascular mortality in Russia present a specific case among developed countries. Application of cardiovascular risk prediction models holds great potential for primary prevention in this country. Using a unique set of cohort follow-up data from Moscow and Saint Petersburg, this study aims to test and recalibrate the Systematic Coronary Risk Evaluation (SCORE) methods for predicting CVD mortality risks in the general population. The study is based on pooled epidemiological cohort data covering the period 1975-2001. The algorithms from the SCORE project were used for the calibration of the SCORE equation for the Moscow and St. Petersburg populations (SCORE-MoSP). Age-specific 10-year cumulative cardiovascular mortality rates were estimated according to the original SCORE-High and SCORE-Low equations and compared to the estimates based on the recalibrated SCORE-MoSP model and observed CVD mortality rates. Ten-year risk prediction charts for CVD mortality were derived and compared using conventional SCORE-High and recalibrated SCORE-MoSP methods. The original SCORE-High model tends to substantially under-estimate 10-year cardiovascular mortality risk for females. The SCORE-MoSP model provided better results which were closer to the observed rates. For males, both the SCORE-High and SCORE-MoSP provided similar estimates which tend to under-estimate CVD mortality risk at younger ages. These differences are also reflected in the risk prediction charts. Using non-calibrated scoring models for Russia may lead to substantial under-estimation of cardiovascular mortality risk in some groups of individuals. Although the SCORE-MoSP provide better results for females, more complex scoring methods involving a wider range of risk factors are needed.
Project description:Cardiovascular disease (CVD) risk prediction tools are often applied to populations beyond those in which they were designed when validated tools for specific subpopulations are unavailable.?Using data from 2283 human immunodeficiency virus (HIV)-infected adults aged ?18 years, who were active in the HIV Outpatient Study (HOPS), we assessed performance of 3 commonly used CVD prediction models developed for general populations: Framingham general cardiovascular Risk Score (FRS), American College of Cardiology/American Heart Association Pooled Cohort equations (PCEs), and Systematic COronary Risk Evaluation (SCORE) high-risk equation, and 1 model developed in HIV-infected persons: the Data Collection on Adverse Effects of Anti-HIV Drugs (D:A:D) study equation. C-statistics assessed model discrimination and the ratio of expected to observed events (E/O) and Hosmer-Lemeshow ?2 P value assessed calibration.?From January 2002 through September 2013, 195 (8.5%) HOPS participants experienced an incident CVD event in 15 056 person-years. The FRS demonstrated moderate discrimination and was well calibrated (C-statistic: 0.66, E/O: 1.01, P = .89). The PCE and D:A:D risk equations demonstrated good discrimination but were less well calibrated (C-statistics: 0.71 and 0.72 and E/O: 0.88 and 0.80, respectively; P < .001 for both), whereas SCORE performed poorly (C-statistic: 0.59, E/O: 1.72; P = .48).?Only the FRS accurately estimated risk of CVD events, while PCE and D:A:D underestimated risk. Although these models could potentially be used to rank US HIV-infected individuals at higher or lower risk for CVD, the models may fail to identify substantial numbers of HIV-infected persons with elevated CVD risk who could potentially benefit from additional medical treatment.
Project description:AIMS:Cardiovascular disease (CVD) risk prediction models are used in Western European countries, but less so in Eastern European countries where rates of CVD can be two to four times higher. We recalibrated the SCORE prediction model for three Eastern European countries and evaluated the impact of adding seven behavioural and psychosocial risk factors to the model. METHODS AND RESULTS:We developed and validated models using data from the prospective HAPIEE cohort study with 14?598 participants from Russia, Poland, and the Czech Republic (derivation cohort, median follow-up 7.2 years, 338 fatal CVD cases) and Estonian Biobank data with 4632 participants (validation cohort, median follow-up 8.3 years, 91 fatal CVD cases). The first model (recalibrated SCORE) used the same risk factors as in the SCORE model. The second model (HAPIEE SCORE) added education, employment, marital status, depression, body mass index, physical inactivity, and antihypertensive use. Discrimination of the original SCORE model (C-statistic 0.78 in the derivation and 0.83 in the validation cohorts) was improved in recalibrated SCORE (0.82 and 0.85) and HAPIEE SCORE (0.84 and 0.87) models. After dichotomizing risk at the clinically meaningful threshold of 5%, and when comparing the final HAPIEE SCORE model against the original SCORE model, the net reclassification improvement was 0.07 [95% confidence interval (CI) 0.02-0.11] in the derivation cohort and 0.14 (95% CI 0.04-0.25) in the validation cohort. CONCLUSION:Our recalibrated SCORE may be more appropriate than the conventional SCORE for some Eastern European populations. The addition of seven quick, non-invasive, and cheap predictors further improved prediction accuracy.
Project description:The 2013 ACC/AHA guidelines introduced an algorithm for risk assessment of atherosclerotic cardiovascular disease (ASCVD) within 10 years. In Germany, risk assessment with the ESC SCORE is limited to cardiovascular mortality. Applicability of the novel ACC/AHA risk score to the German population has not yet been assessed. We therefore sought to recalibrate and evaluate the ACC/AHA risk score in two German cohorts and to compare it to the ESC SCORE.We studied 5,238 participants from the KORA surveys S3 (1994-1995) and S4 (1999-2001) and 4,208 subjects from the Heinz Nixdorf Recall (HNR) Study (2000-2003). There were 383 (7.3%) and 271 (6.4%) first non-fatal or fatal ASCVD events within 10 years in KORA and in HNR, respectively. Risk scores were evaluated in terms of calibration and discrimination performance.The original ACC/AHA risk score overestimated 10-year ASCVD rates by 37% in KORA and 66% in HNR. After recalibration, miscalibration diminished to 8% underestimation in KORA and 12% overestimation in HNR. Discrimination performance of the ACC/AHA risk score was not affected by the recalibration (KORA: C = 0.78, HNR: C = 0.74). The ESC SCORE overestimated by 5% in KORA and by 85% in HNR. The corresponding C-statistic was 0.82 in KORA and 0.76 in HNR.The recalibrated ACC/AHA risk score showed strongly improved calibration compared to the original ACC/AHA risk score. Predicting only cardiovascular mortality, discrimination performance of the commonly used ESC SCORE remained somewhat superior to the ACC/AHA risk score. Nevertheless, the recalibrated ACC/AHA risk score may provide a meaningful tool for estimating 10-year risk of fatal and non-fatal cardiovascular disease in Germany.