Dataset Information

The use of one-stage meta-analytic method based on individual participant data for binary adverse events under the rule of three: a simulation study.

ABSTRACT:

Objective

In evidence synthesis practice, dealing with binary rare adverse events (AEs) is a challenging problem. The pooled estimates for rare AEs through traditional inverse variance (IV), Mantel-Haenszel (MH), and Yusuf-Peto (Peto) methods are suboptimal, as the biases tend to be large. We proposed the "one-stage" approach based on multilevel variance component logistic regression (MVCL) to handle this problem.

Methods

We used simulations to generate trials of individual participant data (IPD) with a series of predefined parameters. We compared the performance of the MVCL "one-stage" approach and the five classical methods (fixed/random effect IV, fixed/random effect MH, and Peto) for rare binary AEs under different scenarios, which included different sample size setting rules, effect sizes, between-study heterogeneity, and numbers of studies in each meta-analysis. The percentage bias, mean square error (MSE), coverage probability, and average width of the 95% confidence intervals were used as performance indicators.

Results

We set 52 scenarios and each scenario was simulated 1,000 times. Under the rule of three (a sample size setting rule to ensure a 95% chance of detecting at least one AE case), the MVCL "one-stage" IPD method had the lowest percentage bias in most of the situations and the bias remained at a very low level (<10%), when compared to IV, MH, and Peto methods. In addition, the MVCL "one-stage" IPD method generally had the lowest MSE and the narrowest average width of 95% confidence intervals. However, it did not show better coverage probability over the other five methods.

Conclusions

The MVCL "one-stage" IPD meta-analysis is a useful method to handle binary rare events and superior compared to traditional methods under the rule of three. Further meta-analyses may take account of the "one-stage" IPD method for pooling rare event data.

SUBMITTER: Cheng LL

PROVIDER: S-EPMC6347966 | biostudies-literature | 2019

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

The use of one-stage meta-analytic method based on individual participant data for binary adverse events under the rule of three: a simulation study.

Cheng Liang-Liang LL Ju Ke K Cai Rui-Lie RL Xu Chang C

PeerJ 20190123

<h4>Objective</h4>In evidence synthesis practice, dealing with binary rare adverse events (AEs) is a challenging problem. The pooled estimates for rare AEs through traditional inverse variance (IV), Mantel-Haenszel (MH), and Yusuf-Peto (Peto) methods are suboptimal, as the biases tend to be large. We proposed the "one-stage" approach based on multilevel variance component logistic regression (MVCL) to handle this problem.<h4>Methods</h4>We used simulations to generate trials of individual partic ...[more]

PMID: 30697486

Similar Datasets

Project description:BackgroundA fundamental aspect of epidemiological studies concerns the estimation of factor-outcome associations to identify risk factors, prognostic factors and potential causal factors. Because reliable estimates for these associations are important, there is a growing interest in methods for combining the results from multiple studies in individual participant data meta-analyses (IPD-MA). When there is substantial heterogeneity across studies, various random-effects meta-analysis models are possible that employ a one-stage or two-stage method. These are generally thought to produce similar results, but empirical comparisons are few.ObjectiveWe describe and compare several one- and two-stage random-effects IPD-MA methods for estimating factor-outcome associations from multiple risk-factor or predictor finding studies with a binary outcome. One-stage methods use the IPD of each study and meta-analyse using the exact binomial distribution, whereas two-stage methods reduce evidence to the aggregated level (e.g. odds ratios) and then meta-analyse assuming approximate normality. We compare the methods in an empirical dataset for unadjusted and adjusted risk-factor estimates.ResultsThough often similar, on occasion the one-stage and two-stage methods provide different parameter estimates and different conclusions. For example, the effect of erythema and its statistical significance was different for a one-stage (OR = 1.35, [Formula: see text]) and univariate two-stage (OR = 1.55, [Formula: see text]). Estimation issues can also arise: two-stage models suffer unstable estimates when zero cell counts occur and one-stage models do not always converge.ConclusionWhen planning an IPD-MA, the choice and implementation (e.g. univariate or multivariate) of a one-stage or two-stage method should be prespecified in the protocol as occasionally they lead to different conclusions about which factors are associated with outcome. Though both approaches can suffer from estimation challenges, we recommend employing the one-stage method, as it uses a more exact statistical approach and accounts for parameter correlation.

Project description:BackgroundResearchers and funders should consider the statistical power of planned Individual Participant Data (IPD) meta-analysis projects, as they are often time-consuming and costly. We propose simulation-based power calculations utilising a two-stage framework, and illustrate the approach for a planned IPD meta-analysis of randomised trials with continuous outcomes where the aim is to identify treatment-covariate interactions.MethodsThe simulation approach has four steps: (i) specify an underlying (data generating) statistical model for trials in the IPD meta-analysis; (ii) use readily available information (e.g. from publications) and prior knowledge (e.g. number of studies promising IPD) to specify model parameter values (e.g. control group mean, intervention effect, treatment-covariate interaction); (iii) simulate an IPD meta-analysis dataset of a particular size from the model, and apply a two-stage IPD meta-analysis to obtain the summary estimate of interest (e.g. interaction effect) and its associated p-value; (iv) repeat the previous step (e.g. thousands of times), then estimate the power to detect a genuine effect by the proportion of summary estimates with a significant p-value.ResultsIn a planned IPD meta-analysis of lifestyle interventions to reduce weight gain in pregnancy, 14 trials (1183 patients) promised their IPD to examine a treatment-BMI interaction (i.e. whether baseline BMI modifies intervention effect on weight gain). Using our simulation-based approach, a two-stage IPD meta-analysis has < 60% power to detect a reduction of 1 kg weight gain for a 10-unit increase in BMI. Additional IPD from ten other published trials (containing 1761 patients) would improve power to over 80%, but only if a fixed-effect meta-analysis was appropriate. Pre-specified adjustment for prognostic factors would increase power further. Incorrect dichotomisation of BMI would reduce power by over 20%, similar to immediately throwing away IPD from ten trials.ConclusionsSimulation-based power calculations could inform the planning and funding of IPD projects, and should be used routinely.

Project description:ImportanceChronic kidney disease (low estimated glomerular filtration rate [eGFR] or albuminuria) affects approximately 14% of adults in the US.ObjectiveTo evaluate associations of lower eGFR based on creatinine alone, lower eGFR based on creatinine combined with cystatin C, and more severe albuminuria with adverse kidney outcomes, cardiovascular outcomes, and other health outcomes.Design, setting, and participantsIndividual-participant data meta-analysis of 27 503 140 individuals from 114 global cohorts (eGFR based on creatinine alone) and 720 736 individuals from 20 cohorts (eGFR based on creatinine and cystatin C) and 9 067 753 individuals from 114 cohorts (albuminuria) from 1980 to 2021.ExposuresThe Chronic Kidney Disease Epidemiology Collaboration 2021 equations for eGFR based on creatinine alone and eGFR based on creatinine and cystatin C; and albuminuria estimated as urine albumin to creatinine ratio (UACR).Main outcomes and measuresThe risk of kidney failure requiring replacement therapy, all-cause mortality, cardiovascular mortality, acute kidney injury, any hospitalization, coronary heart disease, stroke, heart failure, atrial fibrillation, and peripheral artery disease. The analyses were performed within each cohort and summarized with random-effects meta-analyses.ResultsWithin the population using eGFR based on creatinine alone (mean age, 54 years [SD, 17 years]; 51% were women; mean follow-up time, 4.8 years [SD, 3.3 years]), the mean eGFR was 90 mL/min/1.73 m2 (SD, 22 mL/min/1.73 m2) and the median UACR was 11 mg/g (IQR, 8-16 mg/g). Within the population using eGFR based on creatinine and cystatin C (mean age, 59 years [SD, 12 years]; 53% were women; mean follow-up time, 10.8 years [SD, 4.1 years]), the mean eGFR was 88 mL/min/1.73 m2 (SD, 22 mL/min/1.73 m2) and the median UACR was 9 mg/g (IQR, 6-18 mg/g). Lower eGFR (whether based on creatinine alone or based on creatinine and cystatin C) and higher UACR were each significantly associated with higher risk for each of the 10 adverse outcomes, including those in the mildest categories of chronic kidney disease. For example, among people with a UACR less than 10 mg/g, an eGFR of 45 to 59 mL/min/1.73 m2 based on creatinine alone was associated with significantly higher hospitalization rates compared with an eGFR of 90 to 104 mL/min/1.73 m2 (adjusted hazard ratio, 1.3 [95% CI, 1.2-1.3]; 161 vs 79 events per 1000 person-years; excess absolute risk, 22 events per 1000 person-years [95% CI, 19-25 events per 1000 person-years]).Conclusions and relevanceIn this retrospective analysis of 114 cohorts, lower eGFR based on creatinine alone, lower eGFR based on creatinine and cystatin C, and more severe UACR were each associated with increased rates of 10 adverse outcomes, including adverse kidney outcomes, cardiovascular diseases, and hospitalizations.

Project description:BackgroundIndividual participant data meta-analysis (IPD-MA) is considered the gold standard for investigating subgroup effects. Frequently used regression-based approaches to detect subgroups in IPD-MA are: meta-regression, per-subgroup meta-analysis (PS-MA), meta-analysis of interaction terms (MA-IT), naive one-stage IPD-MA (ignoring potential study-level confounding), and centred one-stage IPD-MA (accounting for potential study-level confounding). Clear guidance on the analyses is lacking and clinical researchers may use approaches with suboptimal efficiency to investigate subgroup effects in an IPD setting. Therefore, our aim is to overview and compare the aforementioned methods, and provide recommendations over which should be preferred.MethodsWe conducted a simulation study where we generated IPD of randomised trials and varied the magnitude of subgroup effect (0, 25, 50% relative reduction), between-study treatment effect heterogeneity (none, medium, large), ecological bias (none, quantitative, qualitative), sample size (50,100,200), and number of trials (5,10) for binary, continuous and time-to-event outcomes. For each scenario, we assessed the power, false positive rate (FPR) and bias of aforementioned five approaches.ResultsNaive and centred IPD-MA yielded the highest power, whilst preserving acceptable FPR around the nominal 5% in all scenarios. Centred IPD-MA showed slightly less biased estimates than naïve IPD-MA. Similar results were obtained for MA-IT, except when analysing binary outcomes (where it yielded less power and FPR < 5%). PS-MA showed similar power as MA-IT in non-heterogeneous scenarios, but power collapsed as heterogeneity increased, and decreased even more in the presence of ecological bias. PS-MA suffered from too high FPRs in non-heterogeneous settings and showed biased estimates in all scenarios. Meta-regression showed poor power (< 20%) in all scenarios and completely biased results in settings with qualitative ecological bias.ConclusionsOur results indicate that subgroup detection in IPD-MA requires careful modelling. Naive and centred IPD-MA performed equally well, but due to less bias of the estimates in the presence of ecological bias, we recommend the latter.

Dataset Information

The use of one-stage meta-analytic method based on individual participant data for binary adverse events under the rule of three: a simulation study.

Objective

Methods

Results

Conclusions

Publications

The use of one-stage meta-analytic method based on individual participant data for binary adverse events under the rule of three: a simulation study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets