Quality of reporting in systematic reviews of adverse events: systematic review.
ABSTRACT: OBJECTIVES: To examine the quality of reporting of harms in systematic reviews, and to determine the need for a reporting guideline specific for reviews of harms. DESIGN: Systematic review. DATA SOURCES: Cochrane Database of Systematic Reviews (CDSR) and Database of Abstracts of Reviews of Effects (DARE). REVIEW METHODS: Databases were searched for systematic reviews having an adverse event as the main outcome, published from January 2008 to April 2011. Adverse events included an adverse reaction, harms, or complications associated with any healthcare intervention. Articles with a primary aim to investigate the complete safety profile of an intervention were also included. We developed a list of 37 items to measure the quality of reporting on harms in each review; data were collected as dichotomous outcomes ("yes" or "no" for each item). RESULTS: Of 4644 reviews identified, 309 were systematic reviews or meta-analyses primarily assessing harms (13 from CDSR; 296 from DARE). Despite a short time interval, the comparison between the years of 2008 and 2010-11 showed no difference on the quality of reporting over time (P=0.079). Titles in fewer than half the reviews (proportion of reviews 0.46 (95% confidence interval 0.40 to 0.52)) did not mention any harm related terms. Almost one third of DARE reviews (0.26 (0.22 to 0.31)) did not clearly define the adverse events reviewed, nor did they specify the study designs selected for inclusion in their methods section. Almost half of reviews (n=170) did not consider patient risk factors or length of follow-up when reviewing harms of an intervention. Of 67 reviews of complications related to surgery or other procedures, only four (0.05 (0.01 to 0.14)) reported professional qualifications of the individuals involved. The overall, unweighted, proportion of reviews with good reporting was 0.56 (0.55 to 0.57); corresponding proportions were 0.55 (0.53 to 0.57) in 2008, 0.55 (0.54 to 0.57) in 2009, and 0.57 (0.55 to 0.58) in 2010-11. CONCLUSION: Systematic reviews compound the poor reporting of harms data in primary studies by failing to report on harms or doing so inadequately. Improving reporting of adverse events in systematic reviews is an important step towards a balanced assessment of an intervention.
Project description:<h4>Background</h4>Although the methods for conducting systematic reviews of efficacy are well established, there is much less guidance on how systematic reviews of adverse effects should be performed.<h4>Methods</h4>In order to determine where methodological research is most needed to improve systematic reviews of adverse effects of health care interventions, we conducted a descriptive analysis of systematic reviews published between 1994 and 2005. We searched the Database of Abstracts of Reviews of Effects (DARE) and The Cochrane Database of Systematic Reviews (CDSR) to identify systematic reviews in which the primary outcome was an adverse effect or effects. We then extracted data on many of the elements of the systematic review process including: types of interventions studied, adverse effects of interest, resources searched, search strategies, data sources included in reviews, quality assessment of primary data, nature of the data analysis, and source of funding.<h4>Results</h4>256 reviews were included in our analysis, of which the majority evaluated drug interventions and pre-specified the adverse effect or effects of interest. A median of 3 resources were searched for each review and very few reviews (13/256) provided sufficient information to reproduce their search strategies. Although more than three quarters (185/243) of the reviews sought to include data from sources other than randomised controlled trials, fewer than half (106/256) assessed the quality of the studies that were included. Data were pooled quantitatively in most of the reviews (165/256) but heterogeneity was not always considered. Less than half (123/256) of the reviews reported on the source of funding.<h4>Conclusion</h4>There is an obvious need to improve the methodology and reporting of systematic reviews of adverse effects. The methodology around identification and quality assessment of primary data is the main concern.
Project description:<h4>Objectives</h4>To assess discrepancies in the analyzed outcomes between protocols and published reviews within Cochrane oral health systematic reviews (COHG) on the Cochrane Database of Systematic Reviews (CDSR).<h4>Study design and setting</h4>All COHG systematic reviews on the CDSR and the corresponding protocols were retrieved in November 2014 and information on the reported outcomes was recorded. Data was collected at the systematic review level by two reviewers independently.<h4>Results</h4>One hundred and fifty two reviews were included. In relation to primary outcomes, 11.2% were downgraded to secondary outcomes, 9.9% were omitted altogether in the final publication and new primary outcomes were identified in 18.4% of publications. For secondary outcomes, 2% were upgraded to primary, 12.5% were omitted and 30.9% were newly introduced in the publication. Overall, 45.4% of reviews had at least one discrepancy when compared to the protocol; these were reported in 14.5% reviews. The number of review updates appears to be associated with discrepancies between final review and protocol (OR: 3.18, 95% CI: 1.77, 5.74, p<0.001). The risk of reporting significant results was lower for both downgraded outcomes [RR: 0.52, 95% CI: 0.17, 1.58, p = 0.24] and upgraded or newly introduced outcomes [RR: 0.77, 95% CI: 0.36, 1.64, p = 0.50] compared to outcomes with no discrepancies. The risk of reporting significant results was higher for upgraded or newly introduced outcomes compared to downgraded outcomes (RR = 1.19, 95% CI: 0.65, 2.16, p = 0.57). None of the comparisons reached statistical significance.<h4>Conclusion</h4>While no evidence of selective outcome reporting was found in this study, based on the present analysis of SRs published within COHG systematic reviews, discrepancies between outcomes in pre-published protocols and final reviews continue to be common. Solutions such as the use of standardized outcomes to reduce the prevalence of this issue may need to be explored.
Project description:Despite favourable results from past meta-analyses, some recent large trials have not found heart failure (HF) disease management programs to be beneficial. To explore reasons for this, we evaluated evidence from existing meta-analyses.Systematic review incorporating meta-review was used. We selected meta-analyses of randomized controlled trials published after 1995 in English that examined the effects of HF disease management programs on key outcomes. Databases searched: MEDLINE, EMBASE, Cochrane Database of Systematic Reviews (CDSR), DARE, NHS EED, NHS HTA, Ageline, AMED, Scopus, Web of Science and CINAHL; cited references, experts and existing reviews were also searched.15 meta-analyses were identified containing a mean of 18.5 randomized trials of HF interventions +/- 10.1 (range: 6 to 36). Overall quality of the meta-analyses was very mixed (Mean AMSTAR Score = 6.4 +/- 1.9; range 2-9). Reporting inadequacies were widespread around populations, intervention components, settings and characteristics, comparison, and comparator groups. Heterogeneity (statistical, clinical, and methodological) was not taken into account sufficiently when drawing conclusions from pooled analyses.Meta-analyses of heart failure disease management programs have promising findings but often fail to report key characteristics of populations, interventions, and comparisons. Existing reviews are of mixed quality and do not adequately take account of program complexity and heterogeneity.
Project description:OBJECTIVES:To assess the effect of antihypertensive treatment in the 130-140 mm Hg systolic blood pressure range. DESIGN:Systematic review and meta-analysis. INFORMATION SOURCES:PubMed, CDSR and DARE were searched for the systematic reviews, which were manually browsed for clinical trials. PubMed and Cochrane Central Register of Controlled Trials were searched for trials directly in February 2018. ELIGIBILITY CRITERIA:Randomised double-blind trials with ≥1000 patient-years of follow-up, comparing any antihypertensive agent against placebo. DATA EXTRACTION AND RISK OF BIAS:Two reviewers extracted study-level data, and assessed risk of bias using Cochrane Collaborations risk of bias assessment tool, independently. MAIN OUTCOMES AND MEASURES:Primary outcomes were all-cause mortality, major cardiovascular events and discontinuation due to adverse events. Secondary outcomes were cardiovascular mortality, myocardial infarction, stroke, heart failure, hypotension-related adverse events and renal impairment. RESULTS:Eighteen trials, including 92 567 participants (34% women, mean age 63 years), fulfilled the inclusion criteria. Primary preventive antihypertensive treatment was associated with a neutral effect on all-cause mortality (relative risk 1.00, 95% CI 0.95 to 1.06) and major cardiovascular events (1.01, 0.96 to 1.06), but an increased risk of discontinuation due to adverse events (1.23, 1.03 to 1.47). None of the secondary efficacy outcomes were significantly reduced, but the risk of hypotension-related adverse events increased with treatment (1.71, 1.32 to 2.22). In coronary artery disease secondary prevention, antihypertensive treatment was associated with reduced risk of all-cause mortality (0.91, 0.83 to 0.99) and major cardiovascular events (0.85, 0.77 to 0.94), but doubled the risk of adverse events leading to discontinuation (2.05, 1.62 to 2.61). CONCLUSION:Primary preventive blood pressure lowering in the 130-140 mm Hg systolic blood pressure range adds no cardiovascular benefit, but increases the risk of adverse events. In the secondary prevention, benefits should be weighed against harms. PROSPERO REGISTRATION NUMBER:CRD42018088642.
Project description:Burden of disease should impact research prioritisation.To analyse the Cochrane Database of Systematic Reviews (CDSR) and determine whether systematic reviews and protocols accurately represent disease burden, as measured by disability-adjusted life years (DALYs) from the Global Burden of Disease (GBD) 2010 Study.Two investigators collected GBD disability metrics for 12 external causes of injury in the GBD 2010 Study. These external causes were then assessed for systematic review and protocol representation in CDSR. Data was collected during the month of April 2015. There were no study participants aside from the researchers. Percentage of total 2010 DALYs, 2010 DALY rank, and median DALY percent change from 1990 to 2010 of the 12 external causes of injury were compared with CDSR representation of systematic reviews and protocols. Data were analysed for correlation using Spearman rank correlation.Eleven of the 12 causes were represented by at least one systematic review or protocol in CDSR; the category collective violence and legal intervention had no representation in CDSR. Correlation testing revealed a strong positive correlation that was statistically significant. Representation of road injury; interpersonal violence; fire, heat, and hot substances; mechanical forces; poisonings, adverse effect of medical treatment, and animal contact was well aligned with respect to DALY. Representation of falls was greater compared to DALY, while self-harm, exposure to forces of nature, and other transport injury representation was lower compared to DALY.CDSR representation of external causes of injury strongly correlates with disease burden. The number of systematic reviews and protocols was well aligned for seven out of 12 causes of injury. These results provide high-quality and transparent data that may guide future prioritisation decisions.
Project description:BACKGROUND:Myopia is a common visual disorder with increasing prevalence. Halting progression of myopia is critical, as high myopia can be complicated by a number of vision-compromising conditions. METHODS:Literature search was conducted in the following databases: Medical Literature Analysis and Retrieval System Online (MEDLINE), Excerpta Medica dataBASE (EMBASE), Cochrane Database of Systematic Reviews (CDSR), Database of Abstracts of Reviews of Effects (DARE) and Centre for Reviews and Dissemination (CRD) Health Technology Assessment (HTA) database. Systematic reviews and meta-analyses investigating the efficacy and safety of multiple myopia interventions vs control conditions, were considered. Methodological quality and quality of evidence of eligible studies were assessed using the ROBIS tool and GRADE rating. The degree of overlapping of index publications in the eligible reviews was calculated with the corrected covered area (CCA). RESULTS:Forty-four unique primary studies contained in 18 eligible reviews and involving 6400 children were included in the analysis. CCA was estimated as 6.2% and thus considered moderate. Results demonstrated the superior efficacy of atropine eyedrops; 1% atropine vs placebo (change in refraction: -0.78D, [-?1.30 to -?0.25] in 1 year), 0.025 to 0.05% atropine vs control (change in refraction: -0.51D, [-?0.60 to -?0.41] in 1 year), 0.01% atropine vs control (change in refraction: -0.50D, [-?0.76 to -?0.24] in 1 year). Atropine was followed by orthokeratology (axial elongation: -?0.19?mm, [-?0.21 to -?0.16] in 1 year) and novel multifocal soft contact lenses (change in refraction: -0.15D, [-?0.27 to -?0.03] in 1 year). As regards adverse events, 1% atropine induced blurred near vision (odds ratio [OR] 9.47, [1.17 to 76.78]) and hypersensitivity reactions (OR 8.91, [1.04 to 76.03]). CONCLUSIONS:Existing evidence has failed to convince doctors to uniformly embrace treatments for myopic progression control, possibly due to existence of some heterogeneity, reporting of side effects and lack of long-term follow-up. Research geared towards efficient interventions is still necessary.
Project description:The delivery of optimal medical care to children is dependent on the availability of child relevant research. Our objectives were to: i) systematically review and describe how children are handled in reviews of drug interventions published in the Cochrane Database of Systematic Reviews (CDSR); and ii) determine when effect sizes for the same drug interventions differ between children and adults.We systematically identified all of the reviews relevant to child health in the CDSR 2002, Issue 4. Reviews were included if they investigated the efficacy or effectiveness of a drug intervention for a condition that occurs in both children and adults. Information was extracted on review characteristics including study methods, results, and conclusions.From 1496 systematic reviews, 408 (27%) were identified as relevant to both adult and child health; 52% (213) of these included data from children. No significant differences were found in effect sizes between adults and children for any of the drug interventions or conditions investigated. However, all of the comparisons lacked the power to detect a clinically significant difference and wide confidence intervals suggest important differences cannot be excluded. A large amount of data was unavailable due to inadequate reporting at the trial and systematic review level.Overall, the findings of this study indicate there is a paucity of child-relevant and specific evidence generated from evidence syntheses of drug interventions. The results indicate a need for a higher standard of reporting for participant populations in studies of drug interventions.
Project description:BACKGROUND:There has been increased interest in the role of cannabis for treating medical conditions. The availability of different cannabis-based products can make the side effects of exposure unpredictable. We sought to conduct a scoping review of systematic reviews assessing benefits and harms of cannabis-based medicines for any condition. METHODS:A protocol was followed throughout the conduct of this scoping review. A protocol-guided scoping review conduct. Searches of bibliographic databases (e.g., MEDLINE®, Embase, PsycINFO, the Cochrane Library) and gray literature were performed. Two people selected and charted data from systematic reviews. Categorizations emerged during data synthesis. The reporting of results from systematic reviews was performed at a high level appropriate for a scoping review. RESULTS:After screening 1975 citations, 72 systematic reviews were included. The reviews covered many conditions, the most common being pain management. Several reviews focused on management of pain as a symptom of conditions such as multiple sclerosis (MS), injury, and cancer. After pain, the most common symptoms treated were spasticity in MS, movement disturbances, nausea/vomiting, and mental health symptoms. An assessment of review findings lends to the understanding that, although in a small number of reviews results showed a benefit for reducing pain, the analysis approach and reporting in other reviews was sub-optimal, making it difficult to know how consistent findings are when considering pain in general. Adverse effects were reported in most reviews comparing cannabis with placebo (49/59, 83%) and in 20/24 (83%) of the reviews comparing cannabis to active drugs. Minor adverse effects (e.g., drowsiness, dizziness) were common and reported in over half of the reviews. Serious harms were not as common, but were reported in 21/59 (36%) reviews that reported on adverse effects. Overall, safety data was generally reported study-by-study, with few reviews synthesizing data. Only one review was rated as high quality, while the remaining were rated of moderate (n = 36) or low/critically low (n = 35) quality. CONCLUSIONS:Results from the included reviews were mixed, with most reporting an inability to draw conclusions due to inconsistent findings and a lack of rigorous evidence. Mild harms were frequently reported, and it is possible the harms of cannabis-based medicines may outweigh benefits. SYSTEMATIC REVIEW REGISTRATION:The protocol for this scoping review was posted in the Open Access (https://ruor.uottawa.ca/handle/10393/37247).
Project description:Systematic reviews (SRs) and meta-analyses (MAs) provide the highest possible level of evidence. However, poor conduct or reporting of SRs and MAs may reduce their utility. The PRISMA Statement (Preferred Reporting Items for Systematic reviews and Meta-Analyses) was developed to help authors report their SRs and MAs adequately.Our objectives were to (1) evaluate the quality of reporting of SRs and MAs and their abstracts in otorhinolaryngologic literature using the PRISMA and PRISMA for Abstracts checklists, respectively, (2) compare the quality of reporting of SRs and MAs published in Ear Nose Throat (ENT) journals to the quality of SRs and MAs published in the 'gold standard' Cochrane Database of Systematic Reviews (CDSR), and (3) formulate recommendations to improve reporting of SRs and MAs in ENT journals.On September 3, 2014, we searched the Pubmed database using a combination of filters to retrieve SRs and MAs on otorhinolaryngologic topics published in 2012 and 2013 in the top 5 ENT journals (ISI Web of Knowledge 2013) or CDSR and relevant articles were selected. We assessed how many, and which, PRISMA (for Abstracts) items were reported adequately per journal type.We identified large differences in the reporting of individual items between the two journal types with room for improvement. In general, SRs and MAs published in ENT journals (n = 31) reported a median of 54.4% of the PRISMA items adequately, whereas the 49 articles published in the CDSR reported a median of 100.0 adequately (difference statistically significant, p < 0.001). For abstracts, medians of 41.7% for ENT journals and 75.0% for the CDSR were found (p < 0.001).The reporting of SRs and MAs in ENT journals leaves room for improvement and would benefit if the PRISMA Statement were endorsed by these journals.
Project description:In the EU/EEA, subgroups of international migrants have an increased prevalence of certain infectious diseases. The objective of this study was to examine migrants' acceptability, value placed on outcomes, and accessibility of infectious disease interventions. We conducted a systematic review of qualitative reviews adhering to the PRISMA reporting guidelines. We searched MEDLINE, EMBASE, CINAHL, DARE, and CDSR, and assessed review quality using AMSTAR. We conducted a framework analysis based on the Health Beliefs Model, which was used to organize our preliminary findings with respect to the beliefs that underlie preventive health behavior, including knowledge of risk factors, perceived susceptibility, severity and barriers, and cues to action. We assessed confidence in findings using an adapted GRADE CERQual tool. We included 11 qualitative systematic reviews from 2111 articles. In these studies, migrants report several facilitators to public health interventions. Acceptability depended on migrants' relationship with healthcare practitioners, knowledge of the disease, and degree of disease-related stigma. Facilitators to public health interventions relevant for migrant populations may provide clues for implementation. Trust, cultural sensitivity, and communication skills also have implications for linkage to care and public health practitioner education. Recommendations from practitioners continue to play a key role in the acceptance of infectious disease interventions.