Randomized Controlled Trials Versus Real World Evidence: Neither Magic Nor Myth.
Ontology highlight
ABSTRACT:
SUBMITTER: Eichler HG
PROVIDER: S-EPMC8246742 | biostudies-literature | 2021 May
REPOSITORIES: biostudies-literature
Publications
Randomized Controlled Trials Versus Real World Evidence: Neither Magic Nor Myth.
Eichler Hans-Georg HG Pignatti Francesco F Schwarzer-Daum Brigitte B Hidalgo-Simon Ana A Eichler Irmgard I Arlett Peter P Humphreys Anthony A Vamvakas Spiros S Brun Nikolai N Rasi Guido G
Clinical pharmacology and therapeutics 20201112 5
Compared with drugs from the blockbuster era, recently authorized drugs and those expected in the future present a heterogenous mix of chemicals, biologicals, and cell and gene therapies, a sizable fraction being for rare diseases, and even individualized treatments or individualized combinations. The shift in the nature of products entails secular trends for the definitions of "drugs" and "target population" and for clinical use and evidence generation. We discuss that the lessons learned from ...[more]
Similar Datasets
Project description:BackgroundMany rare events meta-analyses of randomized controlled trials (RCTs) have lower statistical power, and real-world evidence (RWE) is becoming widely recognized as a valuable source of evidence. The purpose of this study is to investigate methods for including RWE in a rare events meta-analysis of RCTs and the impact on the level of uncertainty around the estimates.MethodsFour methods for the inclusion of RWE in evidence synthesis were investigated by applying them to two previously published rare events meta-analyses: the naïve data synthesis (NDS), the design-adjusted synthesis (DAS), the use of RWE as prior information (RPI), and the three-level hierarchical models (THMs). We gauged the effect of the inclusion of RWE by varying the degree of confidence placed in RWE.ResultsThis study showed that the inclusion of RWE in a rare events meta-analysis of RCTs could increase the precision of the estimates, but this depended on the method of inclusion and the level of confidence placed in RWE. NDS cannot consider the bias of RWE, and its results may be misleading. DAS resulted in stable estimates for the two examples, regardless of whether we placed high- or low-level confidence in RWE. The results of the RPI approach were sensitive to the confidence level placed in RWE. The THM was effective in allowing for accommodating differences between study types, while it had a conservative result compared with other methods.ConclusionThe inclusion of RWE in a rare events meta-analysis of RCTs could increase the level of certainty of the estimates and enhance the decision-making process. DAS might be appropriate for inclusion of RWE in a rare event meta-analysis of RCTs, but further evaluation in different scenarios of empirical or simulation studies is still warranted.
| S-EPMC9964527 | biostudies-literature
Project description:BackgroundNumerous observational studies have revealed an increased risk of death and complications with transfusion, but this observation has not been confirmed in randomized controlled trials (RCTs). The "transfusion kills patients" paradox persists in real-world observational studies despite application of analytic methods such as propensity-score matching. We propose a new design to address this long-term existing issue, which if left unresolved, will be deleterious to the healthy generation of evidence that supports optimized transfusion practice.MethodsIn the new design, we stress three aspects for reconciling observational studies and RCTs on transfusion safety: (1) re-definition of the study population according to a stable hemoglobin range (gray zone of transfusion decision; 7.5-9.5 g/dL in this study); (2) selection of comparison groups according to a trigger value (last hemoglobin measurement before transfusion; nadir during hospital stay for control); (3) dealing with patient heterogeneity according to standardized mean difference (SMD) values. We applied the new design to hospitalized older patients (aged ≥60 years) undergoing general surgery at four academic/teaching hospitals. Four datasets were analyzed: a base population before (Base Match-) and after (Base Match+) propensity-score matching to simulate previous observational studies; a study population before (Study Match-) and after (Study Match+) propensity-score matching to demonstrate effects of our design.ResultsOf 6141 older patients, 662 (10.78%) were transfused and showed high heterogeneity compared with those not receiving transfusion, particularly regarding preoperative hemoglobin (mean: 11.0 vs. 13.5 g/dL) and intraoperative bleeding (≥500 mL: 37.9% vs. 2.1%). Patient heterogeneity was reduced with the new design; SMD of the two variables was reduced from approximately 100% (Base Match-) to 0% (Study Match+). Transfusion was related to a higher risk of death and complications in Base Match- (odds ratio [OR], 95% confidence interval [CI]: 2.68, 1.86-3.86) and Base Match+ (2.24, 1.43-3.49), but not in Study Match- (0.77, 0.32-1.86) or Study Match+ (0.66, 0.23-1.89).ConclusionsWe show how choice of study population and analysis could affect real-world study findings. Our results following the new design are in accordance with relevant RCTs, highlighting its value in accelerating the pace of transfusion evidence generation and generalization.
| S-EPMC9724333 | biostudies-literature
Project description:BackgroundWhile randomized controlled trials (RCTs) are based on strict inclusion/exclusion criteria, non-interventional studies (NISs) might provide additional information to guide management in patients more representative to the real-world setting. The aim of this study was to compare baseline characteristics of patients receiving intravitreal treatment in the NIS OCEAN with those from published RCTs.MethodsThe ongoing OCEAN study enrolled patients treated with ranibizumab for neovascular age-related macular degeneration (nAMD), diabetic macular oedema (DME) or branch/central retinal vein occlusion (B/CRVO). Baseline patient characteristics were compared by indication within the OCEAN cohort. Furthermore, the characteristics were set in reference to those of published RCTs in the same indications. Confidence intervals (CIs) were calculated and assessed for statistically significant differences as indicated by non-overlapping CIs.ResultsPatient characteristics in the NIS OCEAN were evaluated for 3,614 patients with nAMD, 1,211 with DME, 204 with BRVO and 121 with CRVO. Between these groups, significant differences in mean age, gender distributions, and mean baseline VA were seen, reflecting known differences between the indications. Compared to the patient characteristics of published RCTs (trials selected by literature search: nAMD: 13 RCTs, DME: 9, RVO: 5), the OCEAN patients' mean age was significantly higher in every indication. The gender distributions across the trials were comparable, with only few differences between OCEAN and the RCTs. Regarding the mean baseline VA, notable differences were found in nAMD and in DME, with VA significantly higher in some RCTs and lower in others.ConclusionsThe described differences underline the complementarity of NISs and RCTs. OCEAN covers a broader spectrum and more variability of patients than do RCTs. As baseline values may have impact on the treatment response (ceiling effect), there is an ongoing need for research in all patient subgroups. Country-specific assessments of patient populations can better reflect the real-world situation. NISs can deliver insights that RCTs may not, as NISs can include non-typical patients, patients with comorbidities, a broader age spectrum and patients of various disease stages.Trial registrationThe NIS OCEAN was registered on www.clinicaltrials.gov (identifier: NCT02194803 ).
| S-EPMC5244516 | biostudies-literature
Project description:Oral corticosteroids (OCS) have long been a mainstay of treatment for asthma exacerbations and chronic severe asthma. However, it is increasingly recognized that both long-term and short-term OCS use are directly associated with a wide range of serious adverse effects, and as such OCS-sparing treatment alternatives are now widely recommended for patients with severe asthma. While several international guidelines recommend these treatments, guidance on OCS tapering, and which patients are most likely to tolerate OCS reduction and/or discontinuation, is still lacking. Several biologics have demonstrated efficacy in patients with OCS-dependent asthma. One OCS-sparing treatment is the anti-interleukin-5 monoclonal antibody mepolizumab, which is approved for the treatment of severe eosinophilic asthma. In addition to improved exacerbation rates, asthma control, quality of life, and lung function among patients with severe eosinophilic asthma, mepolizumab also has an OCS-sparing effect, which has been demonstrated in randomized controlled trials and real-world studies. Both physicians and patients express concerns about the adverse effects of OCS, and additional data from the randomized, controlled SIRIUS trial (NCT01691508) highlight the high level of concern among patients regarding OCS-related burden. In this article, we discuss current guidance on OCS-sparing strategies for patients with severe asthma, provide a summary of the available evidence of the OCS-sparing effect of mepolizumab, and highlight patient and physician perspectives on the use of OCS and OCS-sparing treatments in severe asthma.
| S-EPMC9386863 | biostudies-literature
Project description:Despite the availability of a variety of insulins, rates of insulinisation and the acceptance of insulin therapy is suboptimal in real-world clinical settings. Patient and physician concerns with hypoglycaemia and weight gain are the two key issues that serve to impede appropriate insulinisation in patients with diabetes. Recently introduced second-generation basal insulin analogues [for e.g., insulin glargine 300 U/mL (Gla-300) and insulin degludec] are designed to have improved pharmacokinetic profiles with an intention to deliver steady insulin levels over a longer period. Several randomised controlled and real-world studies have proven the resultant advantages of second-generations insulin analogues in lowering intra-individual variability in plasma insulin levels, flexibility in dosing, a sustained glucose-lowering effect, and decreasing the risk of hypoglycaemia. Gla-300 is one of the newer second-generation basal insulin analogues to have been approved for both type 1 and 2 diabetes. In this article, we review the currently available clinical and real-world data of Gla-300.
| S-EPMC7156297 | biostudies-literature
Project description:Rapid and robust strategies to evaluate the efficacy and effectiveness of novel and existing pharmacotherapeutic interventions (repurposed treatments) in future pandemics are required. Observational "real-world studies" (RWS) can report more quickly than randomized controlled trials (RCTs) and would have value were they to yield reliable results. Both RCTs and RWS were deployed during the coronavirus disease 2019 (COVID-19) pandemic. Comparing results between them offers a unique opportunity to determine the potential value and contribution of each. A learning review of these parallel evidence channels in COVID-19, based on quantitative modeling, can help improve speed and reliability in the evaluation of repurposed therapeutics in a future pandemic. Analysis of all-cause mortality data from 249 observational RWS and RCTs across eight treatment regimens for COVID-19 showed that RWS yield more heterogeneous results, and generally overestimate the effect size subsequently seen in RCTs. This is explained in part by a few study factors: the presence of RWS that are imbalanced for age, gender, and disease severity, and those reporting mortality at 2 weeks or less. Smaller studies of either type contributed negligibly. Analysis of evidence generated sequentially during the pandemic indicated that larger RCTs drive our ability to make conclusive decisions regarding clinical benefit of each treatment, with limited inference drawn from RWS. These results suggest that when evaluating therapies in future pandemics, (1) large RCTs, especially platform studies, be deployed early; (2) any RWS should be large and should have adequate matching of known confounders and long follow-up; (3) reporting standards and data standards for primary endpoints, explanatory factors, and key subgroups should be improved; in addition, (4) appropriate incentives should be in place to enable access to patient-level data; and (5) an overall aggregate view of all available results should be available at any given time.
| S-EPMC10582658 | biostudies-literature
Project description:Tyrosine kinase inhibitors (TKIs) have been improving the prognosis of patients with chronic myeloid leukemia (CML), but there are still large differences in survival among European countries. This raises questions on the added value of results from population-based studies, which use real-world data, compared to results of randomized controlled trials (RCTs) involving patients with CML. There are also questions about the extent of the findings on RCTs effectiveness for patients in the general population. We compare survival data extracted from our previous systematic review and meta-analysis of CML RCTs with the latest updated population-based survival data of EUROCARE-6, the widest collaborative study on cancer survival in Europe. The EUROCARE-6 CML survival estimated in patients (15-64 years) diagnosed in 2000-2006 vs. 2007-2013 revealed that the prognostic improvement highlighted by RCTs was confirmed in real-world settings, too. The study shows, evaluating for the first time all European regions, that the optimal outcome figures obtained in controlled settings for CML are also achievable (and indeed achieved) in real-world settings with prompt introduction of TKIs in daily clinical practice. However, some differences still persist, particularly in Eastern European countries, where overall survival values are lower than elsewhere, probably due to a delayed introduction of TKIs. Our results suggest an insufficient adoption of adequate protocols in daily clinical practice in those countries where CML survival values remain lower in real life than the values obtained in RCTs. New high-resolution population-based studies may help to identify failures in the clinical pathways followed there.
| S-EPMC9333088 | biostudies-literature
Project description:We estimated the frequency of non-specific influenza-associated clinical endpoints to inform the feasibility of pragmatic randomized controlled trials (RCT) assessing relative vaccine effectiveness (rVE). Hospitalization rates of respiratory, cardiovascular and diabetic events were estimated from Denmark and England's electronic databases and stratified by age, comorbidity and influenza vaccination status. We included a seasonal average of 4.5 million Danish and 7.2 million English individuals, 17 and 32% with comorbidities. Annually, approximately 1% of Danish and 0.5% of English individuals were hospitalized for selected events, ~50% of them respiratory. Hospitalization rates were 40-50-fold and 2-10-fold higher in those >50 years and with comorbidities, respectively. Our findings suggest that a pragmatic RCT using non-specific endpoints is feasible. However, for outcomes with rates <2.5%, it would require randomization of ~100,000 participants to have the power to detect a rVE difference of ~13%. Targeting selected groups (older adults, those with comorbidities) where frequency of events is high would improve trial efficiency.
| S-EPMC8866398 | biostudies-literature
Project description:Findings from previous meta-analyses of randomized clinical trials (RCTs) in premature infants with respiratory distress syndrome (RDS) varied as to whether clinical outcomes differed by type of animal-derived pulmonary surfactant; real-world evidence (RWE) was excluded. We extracted study characteristics and outcomes from full-text articles from a systematic search for studies that compared beractant with poractant alfa for RDS in preterm infants. RWE data were tabulated; RCT data were subjected to meta-analyses. Designs, patient characteristics, and follow-up durations varied widely among studies (4 RWE, 15 RCT). RWE studies with adjusted odds ratios (ORs) found no statistically significant between-treatment differences in outcomes. In RCT meta-analyses, no statistically significant between-treatment differences were observed for death (OR [95% confidence interval], 1.35 [0.98-1.86]), bronchopulmonary dysplasia (1.25 [0.96-1.62]), pneumothorax (1.21 [0.72-2.05]), and air leak syndrome (2.28 [0.82-6.39]). Collectively, outcomes were similar with beractant and poractant alfa in RWE studies and pooled RCTs.
| S-EPMC7375954 | biostudies-literature
Project description:Background/aimsThere has been growing interest in better understanding the potential of observational research methods in medical product evaluation and regulatory decision-making. Previously, we used linked claims and electronic health record data to emulate two ongoing randomized controlled trials, characterizing the populations and results of each randomized controlled trial prior to publication of its results. Here, our objective was to compare the populations and results from the emulated trials with those of the now-published randomized controlled trials.MethodsThis study compared participants' demographic and clinical characteristics and study results between the emulated trials, which used structured data from OptumLabs Data Warehouse, and the published PRONOUNCE and GRADE trials. First, we examined the feasibility of implementing the baseline participant characteristics included in the published PRONOUNCE and GRADE trials' using real-world data and classified each variable as ascertainable, partially ascertainable, or not ascertainable. Second, we compared the emulated trials and published randomized controlled trials for baseline patient characteristics (concordance determined using standardized mean differences <0.20) and results of the primary and secondary endpoints (concordance determined by direction of effect estimates and statistical significance).ResultsThe PRONOUNCE trial enrolled 544 participants, and the emulated trial included 2226 propensity score-matched participants. In the PRONOUNCE trial publication, one of the 32 baseline participant characteristics was listed as an exclusion criterion on ClinicalTrials.gov but was ultimately not used. Among the remaining 31 characteristics, 9 (29.0%) were ascertainable, 11 (35.5%) were partially ascertainable, and 10 (32.2%) were not ascertainable using structured data from OptumLabs. For one additional variable, the PRONOUNCE trial did not provide sufficient detail to allow its ascertainment. Of the nine variables that were ascertainable, values in the emulated trial and published randomized controlled trial were discordant for 6 (66.7%). The primary endpoint of time from randomization to the first major adverse cardiovascular event and secondary endpoints of nonfatal myocardial infarction and stroke were concordant between the emulated trial and published randomized controlled trial. The GRADE trial enrolled 5047 participants, and the emulated trial included 7540 participants. In the GRADE trial publication, 8 of 34 (23.5%) baseline participant characteristics were ascertainable, 14 (41.2%) were partially ascertainable, and 11 (32.4%) were not ascertainable using structured data from OptumLabs. For one variable, the GRADE trial did not provide sufficient detail to allow for ascertainment. Of the eight variables that were ascertainable, values in the emulated trial and published randomized controlled trial were discordant for 4 (50.0%). The primary endpoint of time to hemoglobin A1c ≥7.0% was mostly concordant between the emulated trial and the published randomized controlled trial.ConclusionDespite challenges, observational methods and real-world data can be leveraged in certain important situations for a more timely evaluation of drug effectiveness and safety in more diverse and representative patient populations.
| S-EPMC10843567 | biostudies-literature