Selective reporting of outcomes in randomised controlled trials in systematic reviews of cystic fibrosis.
ABSTRACT: Outcome reporting bias (ORB) in randomised trials has been identified as a threat to the validity of systematic reviews. Previous work highlighting this problem is limited to considering a single primary review outcome. The aim of this study was to assess ORB across all efficacy outcomes in the Cochrane systematic reviews of cystic fibrosis.Systematic reviews of interventions for cystic fibrosis published on the Cochrane Library by the Cochrane Cystic Fibrosis and Genetic Disorders Group before 2010 were assessed for discrepancies in outcomes between review protocol and full review. ORB in eligible trials was also assessed for all efficacy review outcomes. Two authors independently classified each outcome using a nine-point classification system developed by the Outcome Reporting Bias In Trials study. These classifications were used to inform the assessment of the risk of bias for selective outcome reporting for each trial.-46 Cochrane cystic fibrosis systematic reviews were included. The median number of primary outcomes, number of trials and participants per trial in the reviews were 3 (IQR 2, 3), 4 (IQR 2, 8) and 21 (IQR 14, 41), respectively. 18 reviews (39%, 18/46) had a discrepancy in outcomes between protocol and full review. 37 reviews were eligible to be included in the ORB assessment. When considering review primary outcomes and all review outcomes, ORB was suspected in at least one trial in 86% and 100%, respectively.Assessment of ORB within a systematic review of a single primary outcome underestimates the risk of ORB in comparison to the assessment of multiple primary and secondary outcomes. ORB in trials is highly prevalent within systematic reviews of cystic fibrosis when assessed across all outcomes. This could be reduced by the development of a core outcome set for trials and systematic reviews in cystic fibrosis.
Project description:BACKGROUND:Discrepancies in outcome reporting (DOR) between protocol and published studies include inclusions of new outcomes, omission of prespecified outcomes, upgrade and downgrade of secondary and primary outcomes, and changes in definitions of prespecified outcomes. DOR can result in outcome reporting bias (ORB) when changes in outcomes occur after knowledge of results. This has potential to overestimate treatment effects and underestimate harms. This can also occur at the level of systematic reviews when changes in outcomes occur after knowledge of results of included studies. The prevalence of DOR and ORB in systematic reviews is unknown in systematic reviews published post-2007. OBJECTIVE:To estimate the prevalence of DOR and risk of ORB in all Cochrane reviews between the years 2007 and 2014. METHODS:A stratified random sampling approach was applied to collect a representative sample of Cochrane systematic reviews from each Cochrane review group. DOR was assessed by matching outcomes in each systematic review with their respective protocol. When DOR occurred, reviews were further assessed if there was a risk of ORB (unclear, low or high risk). We classified DOR as a high risk for ORB if the discrepancy occurred after knowledge of results in the systematic review. RESULTS:150 of 350 (43%) review and protocol pairings contained DOR. When reviews were further scrutinised, 23% (35 of 150) of reviews with DOR contained a high risk of ORB, with changes being made after knowledge of results from individual trials. CONCLUSIONS:In our study, we identified just under a half of Cochrane reviews with at least one DOR. Of these, a fifth were at high risk of ORB. The presence of DOR and ORB in Cochrane reviews is of great concern; however, a solution is relatively simple. Authors are encouraged to be transparent where outcomes change and to describe the legitimacy of changing outcomes in order to prevent suspicion of bias.
Project description:Systematic reviews of treatment interventions in stable or chronic conditions often require the synthesis of clinical trials with a cross-over design. Previous work has indicated that methodology for analysing cross-over data is inadequate in trial reports and in systematic reviews assessing trials with this design.We assessed systematic review methodology for synthesising cross-over trials among Cochrane Cystic Fibrosis and Genetic Disorders Group reviews published to July 2015, and assessed the quality of reporting among the cross-over trials included in these reviews.We performed data extraction of methodology and reporting in reviews, trials identified and trials included within reviews.We reviewed a total of 142 Cochrane systematic reviews including 53 reviews which synthesised evidence from 218 cross-over trials. Thirty-three (63%) Cochrane reviews described a clear and appropriate method for the inclusion of cross-over data, and of these 19 (56%) used the same method to analyse results. 145 cross-over trials were described narratively or treated as parallel trials in reviews but in 30 (21%) of these trials data existed in the trial reports to account for the cross-over design. At the trial level, the analysis and presentation of results were often inappropriate or unclear, with only 69 (32%) trials presenting results that could be included in meta-analysis.Despite development of accessible, technical guidance and training for Cochrane systematic reviewers, statistical analysis and reporting of cross-over data is inadequate at both the systematic review and the trial level. Plain language and practical guidance for the inclusion of cross-over data in meta-analysis would benefit systematic reviewers, who come from a wide range of health specialties. Minimum reporting standards for cross-over trials are needed.
Project description:BACKGROUND:Patients with cystic fibrosis (CF) can struggle with burdensome symptoms and treatment regimens that negatively affect every aspect of their life. As physiological parameters can fail to capture these complications, the assessment of health-related quality of life (HRQOL) has gained prominence. HRQOL can be measured using standardized patient questionnaires called patient-reported outcome measures (PROMs). The Australian Cystic Fibrosis Data Registry (ACFDR) collects clinical data on adult and pediatric patients with CF. The incorporation of PROMs into the ACFDR would enable monitoring of HRQOL trends, benchmarking of HRQOL outcomes, and support of HRQOL research in CF. OBJECTIVE:Prior to incorporation of a PROM in the ACFDR, this systematic review was planned to evaluate whether any suitable PROMs are currently being used for CF. METHODS:This systematic review will be conducted in compliance with the PRISMA-P (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols) guidelines. MEDLINE, EMBASE, Scopus, CINAHL (Cumulative Index of Nursing and Allied Health Literature), PsycINFO, and Cochrane Library databases were searched for articles published between January 2009 and February 2019 on the use of PROMs to measure HRQOL in adult and pediatric patients with CF. Study designs such as observational studies, reviews and validation studies were included. Studies describing randomized controlled trials, dissertations, books, guideline statements, and abstracts were excluded. The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) risk of bias checklist was used to assess the methodological quality of included studies. A descriptive synthesis of the results will be undertaken in line with the outcomes of this study. RESULTS:As of July 2019, the search has been conducted and 4530 records were screened. After two phases of screening, 97 studies were included in the final review and subjected to data extraction. Reviewers are currently in the process of critical appraisal. CONCLUSIONS:This review will identify any PROM(s) that may be used to measure HRQOL in the ACFDR. TRIAL REGISTRATION:PROSPERO International Prospective Register of Systematic Reviews CRD42019126931; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=126931.
Project description:BACKGROUND:Chronic rhinosinusitis frequently occurs in people with cystic fibrosis. Several medical interventions are available for treating chronic rhinosinusitis in people with cystic fibrosis; for example, different concentrations of nasal saline irrigations, topical or oral corticosteroids, antibiotics - including nebulized antibiotics, dornase alfa and modulators of the cystic fibrosis transmembrane conductance regulator (CFTR) (such as lumacaftor, ivacaftor or tezacaftor). However, the efficacy of these interventions is unclear. OBJECTIVES:The objective of this review is to compare the effects of different medical interventions in people diagnosed with cystic fibrosis and chronic rhinosinusitis. SEARCH METHODS:We searched the Cochrane Cystic Fibrosis Trials Register, compiled from electronic database searches and hand searching of journals and conference abstract books. Date of last search of trials register: 22 May 2019.We also searched ongoing trials databases, other medical databases and the reference lists of relevant articles and reviews. Date of latest additional searches: 20 May 2019. SELECTION CRITERIA:Randomized and quasi-randomized trials of different medical interventions compared to each other or to no intervention or to placebo. DATA COLLECTION AND ANALYSIS:Two review authors independently assessed trials identified for potential inclusion in the review. We planned to conduct data collection and analysis in accordance with Cochrane methods and to independently rate the quality of the evidence for each outcome using the GRADE guidelines. MAIN RESULTS:We identified no trials that met the pre-defined inclusion criteria. The searches identified 47 trials, none of which were eligible for inclusion in the current version of this review. AUTHORS' CONCLUSIONS:We identified no eligible trials assessing the medical interventions in people with cystic fibrosis and chronic rhinosinusitis. High-quality trials are needed which should assess the efficacy of different treatment options detailed above for managing chronic rhinosinusitis, preventing pulmonary exacerbations and improving quality of life in people with cystic fibrosis.
Project description:<h4>Objective</h4>To determine the extent and nature of selective non-reporting of harm outcomes in clinical studies that were eligible for inclusion in a cohort of systematic reviews.<h4>Design</h4>Cohort study of systematic reviews from two databases.<h4>Setting</h4>Outcome reporting bias in trials for harm outcomes (ORBIT II) in systematic reviews from the Cochrane Library and a separate cohort of systematic reviews of adverse events.<h4>Participants</h4>92 systematic reviews of randomised controlled trials and non-randomised studies published in the Cochrane Library between issue 9, 2012 and issue 2, 2013 (Cochrane cohort) and 230 systematic reviews published between 1 January 2007 and 31 December 2011 in other publications, synthesising data on harm outcomes (adverse event cohort).<h4>Methods</h4>A 13 point classification system for missing outcome data on harm was developed and applied to the studies.<h4>Results</h4>86% (79/92) of reviews in the Cochrane cohort did not include full data from the main harm outcome of interest of each review for all of the eligible studies included within that review; 76% (173/230) for the adverse event cohort. Overall, the single primary harm outcome was inadequately reported in 76% (705/931) of the studies included in the 92 reviews from the Cochrane cohort and not reported in 47% (4159/8837) of the 230 reviews in the adverse event cohort. In a sample of primary studies not reporting on the single primary harm outcome in the review, scrutiny of the study publication revealed that outcome reporting bias was suspected in nearly two thirds (63%, 248/393).<h4>Conclusions</h4>The number of reviews suspected of outcome reporting bias as a result of missing or partially reported harm related outcomes from at least one eligible study is high. The declaration of important harms and the quality of the reporting of harm outcomes must be improved in both primary studies and systematic reviews.
Project description:BACKGROUND:The antibiotics used to treat pulmonary infections in people with cystic fibrosis are typically chosen based on the results of antimicrobial susceptibility testing performed on bacteria traditionally grown in a planktonic mode (grown in a liquid). However, there is considerable evidence to suggest that Pseudomonas aeruginosa actually grows in a biofilm (or slime layer) in the airways of people with cystic fibrosis with chronic pulmonary infections. Therefore, choosing antibiotics based on biofilm rather than conventional antimicrobial susceptibility testing could potentially improve response to treatment of Pseudomonas aeruginosa in people with cystic fibrosis. This is an update of a previously published Cochrane Review. OBJECTIVES:To compare biofilm antimicrobial susceptibility testing-driven therapy to conventional antimicrobial susceptibility testing-driven therapy in the treatment of Pseudomonas aeruginosa infection in people with cystic fibrosis. SEARCH METHODS:We searched the Cochrane Cystic Fibrosis Trials Register, compiled from electronic database searches and handsearching of journals and conference abstract books. Most recent search: 19 June 2017.We also searched two ongoing trials registries and the reference lists of relevant articles and reviews. Most recent searches: 24 August 2017 and 05 September 2017. SELECTION CRITERIA:Randomized controlled trials of antibiotic therapy based on biofilm antimicrobial susceptibility testing compared to antibiotic therapy based on conventional antimicrobial susceptibility testing in the treatment of Pseudomonas aeruginosa pulmonary infection in people with cystic fibrosis. DATA COLLECTION AND ANALYSIS:Both authors independently selected trials, assessed their risk of bias and extracted data from eligible trials. Additionally, the review authors contacted the trial investigators to obtain further information. The quality of the evidence was assessed using the GRADE criteria. MAIN RESULTS:The searches identified two multicentre, randomized, double-blind controlled clinical trials eligible for inclusion in the review with a total of 78 participants (adults and children); one trial was done in people who were clinically stable, the other in people experiencing pulmonary exacerbations. These trials prospectively assessed whether the use of biofilm antimicrobial susceptibility testing improved microbiological and clinical outcomes in participants with cystic fibrosis who were infected with Pseudomonas aeruginosa. The primary outcome was the change in sputum Pseudomonas aeruginosa density from the beginning to the end of antibiotic therapy.Although the intervention was shown to be safe, the data from these two trials did not provide evidence that biofilm susceptibility testing was superior to conventional susceptibility testing either in terms of microbiological or lung function outcomes. One of the trials also measured risk and time to subsequent exacerbation as well as quality of life measures and did not demonstrate any difference between groups in these outcomes. Both trials had an overall low risk of bias and the quality of the evidence using GRADE criteria was deemed to be moderate to high for the outcomes selected. AUTHORS' CONCLUSIONS:The current evidence is insufficient to recommend choosing antibiotics based on biofilm antimicrobial susceptibility testing rather than conventional antimicrobial susceptibility testing in the treatment of Pseudomonas aeruginosa pulmonary infections in people with cystic fibrosis. Biofilm antimicrobial susceptibility testing may be more appropriate in the development of newer, more effective formulations of drugs which can then be tested in clinical trials.
Project description:There is increasing recognition that insufficient attention has been paid to the choice of outcomes measured in clinical trials. The lack of a standardized outcome classification system results in inconsistencies due to ambiguity and variation in how outcomes are described across different studies. Being able to classify by outcome would increase efficiency in searching sources such as clinical trial registries, patient registries, the Cochrane Database of Systematic Reviews, and the Core Outcome Measures in Effectiveness Trials (COMET) database of core outcome sets (COS), thus aiding knowledge discovery.A literature review was carried out to determine existing outcome classification systems, none of which were sufficiently comprehensive or granular for classification of all potential outcomes from clinical trials. A new taxonomy for outcome classification was developed, and as proof of principle, outcomes extracted from all published COS in the COMET database, selected Cochrane reviews, and clinical trial registry entries were classified using this new system.Application of this new taxonomy to COS in the COMET database revealed that 274/299 (92%) COS include at least one physiological outcome, whereas only 177 (59%) include at least one measure of impact (global quality of life or some measure of functioning) and only 105 (35%) made reference to adverse events.This outcome taxonomy will be used to annotate outcomes included in COS within the COMET database and is currently being piloted for use in Cochrane Reviews within the Cochrane Linked Data Project. Wider implementation of this standard taxonomy in trial and systematic review databases and registries will further promote efficient searching, reporting, and classification of trial outcomes.
Project description:BACKGROUND: There is no consensus on how authors conducting meta-analysis should deal with trial participants with missing outcome data. The objectives of this study are to assess in Cochrane and non-Cochrane systematic reviews: (1) which categories of trial participants the systematic review authors consider as having missing participant data (MPD), (2) how trialists reported on participants with missing outcome data in trials, (3) whether systematic reviewer authors actually dealt with MPD in their meta-analyses of dichotomous outcomes consistently with their reported methods, and (4) the impact of different methods of dealing with MPD on pooled effect estimates in meta-analyses of dichotomous outcomes. METHODS/DESIGN: We will conduct a methodological study of Cochrane and non-Cochrane systematic reviews. Eligible systematic reviews will include a group-level meta-analysis of a patient-important dichotomous efficacy outcome, with a statistically significant effect estimate. Teams of two reviewers will determine eligibility and subsequently extract information from each eligible systematic review in duplicate and independently, using standardized, pre-piloted forms. The teams will then use a similar process to extract information from the trials included in the meta-analyses of interest. We will assess first which categories of trial participants the systematic reviewers consider as having MPD. Second, we will assess how trialists reported on participants with missing outcome data in trials. Third, we will compare what systematic reviewers report having done, and what they actually did, in dealing with MPD in their meta-analysis. Fourth, we will conduct imputation studies to assess the effects of different methods of dealing with MPD on the pooled effect estimates of meta-analyses. We will specifically calculate for each method (1) the percentage of systematic reviews that lose statistical significance and (2) the mean change of effect estimates across systematic reviews. DISCUSSION: The impact of different methods of dealing with MPD on pooled effect estimates will help judge the associated risk of bias in systematic reviews. Our findings will inform recommendations regarding what assumptions for MPD should be used to test the robustness of meta-analytical results.
Project description:Many randomized controlled trials (RCTs) report more than one primary outcome. As a result, multivariate meta-analytic methods for the assimilation of treatment effects in systematic reviews of RCTs have received increasing attention in the literature. These methods show promise with respect to bias reduction and efficiency gain compared to univariate meta-analysis. However, most methods for multivariate meta-analysis have focused on pairwise treatment comparisons (i.e., when the number of treatments is two). Current methods for mixed treatment comparisons (MTC) meta-analysis (i.e., when the number of treatments is more than two) have focused on univariate or very recently, bivariate outcomes. To broaden their application, we propose a framework for MTC meta-analysis of multivariate (? 2) outcomes where the correlations among multivariate outcomes within- and between-studies are accounted for through copulas, and the joint modeling of multivariate random effects, respectively. We consider a Bayesian hierarchical model using Markov Chain Monte Carlo methods for estimation. An important feature of the proposed framework is that it allows for borrowing of information across correlated outcomes. We show via simulation that our approach reduces the impact of outcome reporting bias (ORB) in a variety of missing outcome scenarios. We apply the method to a systematic review of RCTs of pharmacological treatments for alcohol dependence, which tends to report multiple outcomes potentially subject to ORB.
Project description:OBJECTIVES:A core outcome set (COS) is an agreed standardised minimum collection of outcomes that should be measured and reported in research in a specific area of health. Cochrane systematic reviews ('reviews') are rigorous reviews on health-related topics conducted under the auspices of Cochrane. This study examines the use of existing COS to inform the choice of outcomes in Cochrane systematic reviews ('reviews') and investigates the views of the coordinating editors of Cochrane Review Groups (CRGs) on this topic. METHODS:A cohort of 100 recently published or updated Cochrane reviews were assessed for reference to a COS being used to inform the choice of outcomes for the review. Existing COS, published 2 or more years before the review publication, were then identified to assess how often a reviewer could have used a relevant COS if it was available. We asked 52 CRG coordinating editors about their involvement in COS development, how outcomes are selected for reviews in their CRG and their views of the advantages and challenges surrounding the standardisation of outcomes within their CRG. RESULTS:In the cohort of reviews from 2019, 40% (40/100) of reviewers noted problems due to outcome inconsistency across the included studies. In 7% (7/100) of reviews, a COS was referenced in relation to the choice of outcomes for the review. Relevant existing COS could be considered for a review update in 35% of the others (33/93). Most editors who responded (31/36, 86%) thought that COS should definitely or possibly be used to inform the choice of outcomes in a review. CONCLUSIONS:Systematic reviewers are continuing to note outcome heterogeneity but are starting to use COS to inform their reviews. There is potential for greater uptake of COS in Cochrane reviews.