Dataset Information

Agreement was moderate between data-based and opinion-based assessments of biases affecting randomized trials within meta-analyses.

ABSTRACT:

Background and objective

Randomized trials included in meta-analyses are often affected by bias caused by methodological flaws or limitations, but the degree of bias is unknown. Two proposed methods adjust the trial results for bias using empirical evidence from published meta-epidemiological studies or expert opinion.

Methods

We investigated agreement between data-based and opinion-based approaches to assessing bias in each of four domains: sequence generation, allocation concealment, blinding, and incomplete outcome data. From each sampled meta-analysis, a pair of trials with the highest and lowest empirical model-based bias estimates was selected. Independent assessors were asked which trial within each pair was judged more biased on the basis of detailed trial design summaries.

Results

Assessors judged trials to be equally biased in 68% of pairs evaluated. When assessors judged one trial as more biased, the proportion of judgments agreeing with the model-based ranking was highest for allocation concealment (79%) and blinding (79%) and lower for sequence generation (59%) and incomplete outcome data (56%).

Conclusion

Most trial pairs found to be discrepant empirically were judged to be equally biased by assessors. We found moderate agreement between opinion and data-based evidence in pairs where assessors ranked one trial as more biased.

SUBMITTER: Turner RM

PROVIDER: S-EPMC7482431 | biostudies-literature | 2020 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Agreement was moderate between data-based and opinion-based assessments of biases affecting randomized trials within meta-analyses.

Turner Rebecca M RM Rhodes Kirsty M KM Jones Hayley E HE Higgins Julian P T JPT Haskins Jessica A JA Whiting Penny F PF Hróbjartsson Asbjørn A Caldwell Deborah M DM Morris Richard W RW Reeves Barnaby C BC Worthington Helen V HV Boutron Isabelle I Savović Jelena J

Journal of clinical epidemiology 20200513

<h4>Background and objective</h4>Randomized trials included in meta-analyses are often affected by bias caused by methodological flaws or limitations, but the degree of bias is unknown. Two proposed methods adjust the trial results for bias using empirical evidence from published meta-epidemiological studies or expert opinion.<h4>Methods</h4>We investigated agreement between data-based and opinion-based approaches to assessing bias in each of four domains: sequence generation, allocation conceal ...[more]

PMID: 32416338

Similar Datasets

Project description:BackgroundDuring clinical trials, researchers rarely question nominal doses specified on labels of investigational products, overlooking the potential for inaccuracies that may result when calculating pharmacokinetic and pharmacodynamic parameters. This study evaluated the disparity between nominal doses and the doses actually administered in two Phase I trials of a biosimilar drug.MethodsIn Trial A, 12 healthy volunteers received various doses of an interferon β-1a biosimilar via either subcutaneous or intravenous injection, prepared by partially emptying 0.53 ml syringes supplied by the manufacturer. In Trial B, 12 volunteers received three different formulations of the drug via intravenous injection (biosimilar with and without albumin and a comparator), followed by multiple subcutaneous injections. In both trials, the dose administered was calculated as D = C × V - losses, where C is the drug concentration assessed using ELISA, V is the volume administered calculated using syringe weighing and losses are deduced from in-vitro experiments. Interferon binding to added albumin and infusion lines was evaluated using a (125)I-interferon tracer with gel-filtration chromatography.ResultsIn Trial A, measured concentrations were close to the nominal strength indicated by the manufacturer (median bias: -6 %), whereas in Trial B they differed significantly for all three formulations (median biases: +67 %, +73 % and +31 % for the biosimilar with albumin, the biosimilar without albumin and the comparator, respectively). In Trial A, the doses actually administered showed large variability and biases, especially at the lowest doses. Indeed, actually injected volumes differed by as much as 74 % from theoretical volumes - a phenomenon mainly attributed to unnoticed fluid re-aspiration through the syringe needle. This was corrected in Trial B. Interferon was not significantly adsorbed on the infusion lines used for intravenous administration. Its binding to albumin was slow, reaching 50 % after a 16 h incubation.ConclusionsThese examples illustrate the importance of assessing the actual doses administered in clinical trials, to ensure accuracy in the determination of clearance, distribution volume, bioavailability and dose-response relationships.Trial registrationClinicaltrials.gov NCT02515695 (Trial A) and NCT02517788 (Trial B). Registered on 24 July and 5 August 2015, respectively.

Project description:BackgroundMagnetic resonance imaging (MRI)-based rotator cuff assessment is often qualitative and subjective; few studies have tried to validate such preoperative assessments. This study investigates relationships of preoperative MRI assessments made by conventional approaches to intraoperative findings of tear type, location, and size or MRI-assessed muscle occupation ratio.MethodsIntraoperatively, surgeons assessed tear type, location, anterior-posterior (AP) width, and medial-lateral length in 102 rotator cuff repair patients. Two musculoskeletal radiologists independently assessed the preoperative MRI scans for these same parameters and supraspinatus muscle atrophy by both Warner classification and quantitative occupation ratio. Exact agreement proportions, kappa statistics, and correlation coefficients were used to quantify agreement relationships.ResultsAgreement between MRI readers' and surgeons' observations of tear status averaged 93% with κ = 0.38, and that of tear location averaged 77% with κ = 0.50. Concordance correlations of MRI and intraoperative measures of anterior-posterior and medial-lateral tear length averaged 0.59 and 0.56 across readers, respectively. Despite excellent interrater agreement on Warner classification (exact agreement proportion 0.91) and occupation ratio (concordance correlation 0.93) separately, correlations between these 2 measures were -0.54 and -0.64 for the 2 readers, respectively. Patients with Warner grade 0 had occupation ratios ranging from 0.5 to 1.5.ConclusionCorrelations of preoperative MRI tear dimensions and muscle atrophy assessed by conventional approaches with intraoperatively measured tear dimensions and quantitative occupation ratio, respectively, were only fair. Since tear size and muscle atrophy are known strong predictors of outcomes following rotator cuff repair that may influence treatment decisions, surgeons need to be aware of the limitations of MRI methods. Continued development and validation of quantitative preoperative imaging methods to accurately assess these parameters are needed to improve surgical planning and prognosis.

Project description:Skeletal muscle perfusion and oxygenation are commonly evaluated using Doppler ultrasound and near-infrared spectroscopy (NIRS) techniques. However, a recently developed magnetic resonance imaging (MRI) sequence, termed PIVOT, permits the simultaneous collection of skeletal muscle perfusion and T2* (an index of skeletal muscle oxygenation).PurposeTo determine the level of agreement between PIVOT, Doppler ultrasound, and NIRS-based assessments of skeletal muscle perfusion and oxygenation.MethodsTwelve healthy volunteers (8 females, 25 ± 3 years, 170 ± 11 cm, 71.5 ± 8.0 kg) performed six total reactive hyperemia protocols. During three of these reactive hyperemia protocols, Tissue Saturation Index (TSI) and oxygenated hemoglobin (O2Hb) were recorded from the medial gastrocnemius (MG) and tibialis anterior (TA), and blood flow velocity was recorded from the popliteal artery (BFvpop) via Doppler Ultrasound. The other three trials were performed inside the bore of a 3 T MRI scanner, and the PIVOT sequence was used to assess perfusion (PIVOTperf) and oxygenation (T2*) of the medial gastrocnemius and tibialis anterior muscles. Positive incremental areas under the curve (iAUC) and times to peak (TTP) were calculated for each variable, and the level of agreement between collection methods was evaluated via Bland-Altman analyses and Spearman's Rho correlation analyses.ResultsThe only significant bivariate relationships observed were between the T2* vs. TSI iAUC and PIVOTperf vs. BFvpop values recorded from the MG. Significant mean differences were observed for all comparisons (all P ≤ 0.038), and significant proportional biases were observed for the PIVOTperf vs. tHb TTP (R2 = 0.848, P < 0.001) and T2* vs. TSI TTP comparisons in the TA (R2 = 0.488, P = 0.011), and the PIVOTperf vs. BFvpop iAUC (R2 = 0.477, P = 0.013) and time to peak (R2 = 0.851, P < 0.001) comparisons in the MG.ConclusionsOur findings suggest that the PIVOT technique has, at best, a moderate level of agreement with Doppler ultrasound and NIRS assessment methods and is subject to significant proportional bias. These findings do not challenge the accuracy of either measurement technique but instead reflect differences in the vascular compartments, sampling volumes, and parameters being evaluated.

Dataset Information

Agreement was moderate between data-based and opinion-based assessments of biases affecting randomized trials within meta-analyses.

Background and objective

Methods

Results

Conclusion

Publications

Agreement was moderate between data-based and opinion-based assessments of biases affecting randomized trials within meta-analyses.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets