Spin in the reporting, interpretation, and extrapolation of adverse effects of orthodontic interventions: protocol for a cross-sectional study of systematic reviews.
ABSTRACT: Background:Titles and abstracts are the most read sections of biomedical papers. It is therefore important that abstracts transparently report both the beneficial and adverse effects of health care interventions and do not mislead the reader. Misleading reporting, interpretation, or extrapolation of study results is called "spin". In this study, we will assess whether adverse effects of orthodontic interventions were reported or considered in the abstracts of both Cochrane and non-Cochrane reviews and whether spin was identified and what type of spin. Methods:Eligibility criteria were defined for the type of study designs, participants, interventions, outcomes, and settings. We will include systematic reviews of clinical orthodontic interventions published in the five leading orthodontic journals and in the Cochrane Database. Empty reviews will be excluded. We will manually search eligible reviews published between 1 August 2009 and 31 July 2019. Data collection forms were developed a priori. All study selection and data extraction procedures will be conducted by two reviewers independently. Our main outcomes will be the prevalence of reported or considered adverse effects of orthodontic interventions in the abstract of systematic reviews and the prevalence of "spin" related to these adverse effects. We will also record the prevalence of three subtypes of spin, i.e., misleading reporting, misleading interpretation, and misleading extrapolation-related spin. All statistics will be calculated for the following groups: (1) all journals individually, (2) all journals together, and (3) the five leading orthodontic journals and the Cochrane Database of Systematic Reviews separately. Generalized linear models will be developed to compare the various groups. Discussion:We expect that our results will raise the awareness of the importance of reporting and considering of adverse effects and the presence of the phenomenon of spin related to these effects in abstracts of systematic reviews of orthodontic interventions. This is important, because an incomplete and inadequate reporting, interpretation, or extrapolation of findings on adverse effects in abstracts of systematic reviews can mislead readers and could lead to inadequate clinical practice. Our findings could result in policy implications for making judgments about the acceptance for publication of systematic reviews of orthodontic interventions.
Project description:BACKGROUND:Before implementing healthcare interventions, clinicians need to weigh the beneficial and adverse effects of interventions. However, a large body of evidence has demonstrated that seeking and reporting of adverse effects is suboptimal in clinical trials and in systematic reviews of interventions. This cross-sectional study will investigate the status of this problem in orthodontics. This study will assess whether adverse effects were sought and whether findings related to adverse effects were reported in systematic reviews of orthodontic interventions in the five leading orthodontic journals and in the Cochrane Database of Systematic Reviews. METHODS:Systematic reviews of clinical orthodontic interventions published between 01 August 2009 and 31 July 2019 in the five leading orthodontic journals and in the Cochrane Database will be included. Empty reviews will be excluded. The reporting of outcomes on adverse effects will not determine eligibility, i.e., reviews will not be excluded, because they did not report usable data. Study selection and data extraction will be conducted independently by two authors. Our primary outcome will be the prevalence of systematic reviews of orthodontic interventions that sought any findings related to adverse effects in the included studies. Additional prevalence statistics will be calculated on a series of items related to seeking of adverse effects in the eligible reviews. All statistics will be calculated for (1) all journals together, (2) the group of five orthodontic journals and the Cochrane Database of Systematic Reviews separately, and (3) each individual journal separately. Chi-square tests of independence will be used to compare these groups. DISCUSSION:This study will assess whether adverse effects were sought in systematic reviews of orthodontic interventions. This knowledge is important, because reviews that present an incomplete picture on adverse effects can have unfavorable consequences for the end-users. Also not reporting that no adverse effects were assessed in eligible studies included in a systematic review can mislead pertinent stakeholders. Our findings could have policy implications for making judgments on accepting or rejecting an intervention systematic review for publication, for example, by directing editors and peer-reviewers to adopt the various items on adverse effects defined in the MECIR standards and in the PRISMA harm checklist.
Project description:BACKGROUND:Cancer prognostic biomarkers have shown disappointing clinical applicability. The objective of this study was to classify and estimate how study results are overinterpreted and misreported in prognostic factor studies in oncology. METHODS:This systematic review focused on 17 oncology journals with an impact factor above 7. PubMed was searched for primary clinical studies published in 2015, evaluating prognostic factors. We developed a classification system, focusing on three domains: misleading reporting (selective, incomplete reporting, misreporting), misleading interpretation (unreliable statistical analysis, spin) and misleading extrapolation of the results (claiming irrelevant clinical applicability, ignoring uncertainty). RESULTS:Our search identified 10,844 articles. The 98 studies included investigated a median of two prognostic factors (Q1-Q3, 1-7). The prognostic factors' effects were selectively and incompletely reported in 35/98 and 24/98 full texts, respectively. Twenty-nine articles used linguistic spin in the form of strong statements. Linguistic spin rejecting non-significant results was found in 34 full-text results and 15 abstract results sections. One in five articles had discussion and/or abstract conclusions that were inconsistent with the study findings. Sixteen reports had discrepancies between their full-text and abstract conclusions. CONCLUSIONS:Our study provides evidence of frequent overinterpretation of findings of prognostic factor assessment in high-impact medical oncology journals.
Project description:Although the methods for conducting systematic reviews of efficacy are well established, there is much less guidance on how systematic reviews of adverse effects should be performed.In order to determine where methodological research is most needed to improve systematic reviews of adverse effects of health care interventions, we conducted a descriptive analysis of systematic reviews published between 1994 and 2005. We searched the Database of Abstracts of Reviews of Effects (DARE) and The Cochrane Database of Systematic Reviews (CDSR) to identify systematic reviews in which the primary outcome was an adverse effect or effects. We then extracted data on many of the elements of the systematic review process including: types of interventions studied, adverse effects of interest, resources searched, search strategies, data sources included in reviews, quality assessment of primary data, nature of the data analysis, and source of funding.256 reviews were included in our analysis, of which the majority evaluated drug interventions and pre-specified the adverse effect or effects of interest. A median of 3 resources were searched for each review and very few reviews (13/256) provided sufficient information to reproduce their search strategies. Although more than three quarters (185/243) of the reviews sought to include data from sources other than randomised controlled trials, fewer than half (106/256) assessed the quality of the studies that were included. Data were pooled quantitatively in most of the reviews (165/256) but heterogeneity was not always considered. Less than half (123/256) of the reviews reported on the source of funding.There is an obvious need to improve the methodology and reporting of systematic reviews of adverse effects. The methodology around identification and quality assessment of primary data is the main concern.
Project description:OBJECTIVES:To identify and calculate the prevalence of spin in studies of spin. DESIGN:Meta-research analysis (research on research). SETTING:35 studies of spin in the scientific literature. MAIN OUTCOME MEASURES:Spin, categorised as: reporting practices that distort the presentation and interpretation of results, creating misleading conclusions; discordance between results and their interpretation, with presentation of favourable conclusions that are not supported by the data or results; attribution of causality when study design does not support it; and over-interpretation or inappropriate extrapolation of results. RESULTS:Five (14%) of 35 spin studies contained spin categorised as reporting practices that distort the presentation and interpretation of results (n=2) or categorised as over-interpretation or inappropriate extrapolation of results (n=3). CONCLUSION:Spin occurs in research on spin. Although researchers on this topic should be sensitive to spinning their findings, our study does not undermine the need for rigorous interventions to reduce spin across various research fields. CONCLUSION WITH SPIN:Our hypothesis that spin will be less prevalent in spin studies than in studies on other topics has been proven. Spin scholars are less likely to spin their conclusions than other researchers, and they should receive substantial resources to launch and test interventions to reduce spin and research waste in reporting.
Project description:Systematic reviews (SRs) and meta-analyses (MAs) provide the highest possible level of evidence. However, poor conduct or reporting of SRs and MAs may reduce their utility. The PRISMA Statement (Preferred Reporting Items for Systematic reviews and Meta-Analyses) was developed to help authors report their SRs and MAs adequately.Our objectives were to (1) evaluate the quality of reporting of SRs and MAs and their abstracts in otorhinolaryngologic literature using the PRISMA and PRISMA for Abstracts checklists, respectively, (2) compare the quality of reporting of SRs and MAs published in Ear Nose Throat (ENT) journals to the quality of SRs and MAs published in the 'gold standard' Cochrane Database of Systematic Reviews (CDSR), and (3) formulate recommendations to improve reporting of SRs and MAs in ENT journals.On September 3, 2014, we searched the Pubmed database using a combination of filters to retrieve SRs and MAs on otorhinolaryngologic topics published in 2012 and 2013 in the top 5 ENT journals (ISI Web of Knowledge 2013) or CDSR and relevant articles were selected. We assessed how many, and which, PRISMA (for Abstracts) items were reported adequately per journal type.We identified large differences in the reporting of individual items between the two journal types with room for improvement. In general, SRs and MAs published in ENT journals (n = 31) reported a median of 54.4% of the PRISMA items adequately, whereas the 49 articles published in the CDSR reported a median of 100.0 adequately (difference statistically significant, p < 0.001). For abstracts, medians of 41.7% for ENT journals and 75.0% for the CDSR were found (p < 0.001).The reporting of SRs and MAs in ENT journals leaves room for improvement and would benefit if the PRISMA Statement were endorsed by these journals.
Project description:BACKGROUND: Systematic reviews (SRs) of TCM have become increasingly popular in China and have been published in large numbers. This review provides the first examination of epidemiological characteristics of these SRs as well as compliance with the PRISMA and AMSTAR guidelines. OBJECTIVES: To examine epidemiological and reporting characteristics as well as methodological quality of SRs of TCM published in Chinese journals. METHODS: Four Chinese databases were searched (CBM, CSJD, CJFD and Wanfang Database) for SRs of TCM, from inception through Dec 2009. Data were extracted into Excel spreadsheets. The PRISMA and AMSTAR checklists were used to assess reporting characteristics and methodological quality, respectively. RESULTS: A total of 369 SRs were identified, most (97.6%) of which used the terms systematic review or meta-analysis in the title. None of the reviews had been updated. Half (49.8%) were written by clinicians and nearly half (47.7%) were reported in specialty journals. The impact factors of 45.8% of the journals published in were zero. The most commonly treated conditions were diseases of the circulatory and digestive disease. Funding sources were not reported for any reviews. Most (68.8%) reported information about quality assessment, while less than half (43.6%) reported assessing for publication bias. Statistical mistakes appeared in one-third (29.3%) of reviews and most (91.9%) did not report on conflict of interest. CONCLUSIONS: While many SRs of TCM interventions have been published in Chinese journals, the quality of these reviews is troubling. As a potential key source of information for clinicians and researchers, not only were many of these reviews incomplete, some contained mistakes or were misleading. Focusing on improving the quality of SRs of TCM, rather than continuing to publish them in great quantity, is urgently needed in order to increase the value of these studies.
Project description:OBJECTIVE: To examine whether network meta-analyses, increasingly used to assess comparative effectiveness of healthcare interventions, follow the key methodological recommendations for reporting and conduct of systematic reviews. DESIGN: Methodological systematic review of reports of network meta-analyses. DATA SOURCES: Cochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Effects, Medline, and Embase, searched from inception to 12 July 2012. REVIEW METHODS: All network meta-analyses comparing clinical efficacy of three or more interventions based on randomised controlled trials, excluding meta-analyses with an open loop network of three interventions. We assessed the reporting of general characteristics and key methodological components of the systematic review process using two composite outcomes. For some components, if reporting was adequate, we assessed their conduct quality. RESULTS: Of 121 network meta-analyses covering a wide range of medical areas, 100 (83%) assessed pharmacological interventions and 11 (9%) non-pharmacological interventions; 56 (46%) were published in journals with a high impact factor. The electronic search strategy for each database was not reported in 88 (73%) network meta-analyses; for 36 (30%), the primary outcome was not clearly identified. Overall, 61 (50%) network meta-analyses did not report any information regarding the assessment of risk of bias of individual studies, and 103 (85%) did not report any methods to assess the likelihood of publication bias. Overall, 87 (72%) network meta-analyses did not report the literature search, searched only one database, did not search other sources, or did not report an assessment of risk of bias of individual studies. These methodological components did not differ by publication in a general or specialty journal or by public or private funding. CONCLUSIONS: Essential methodological components of the systematic review process-conducting a literature search and assessing risk of bias of individual studies-are frequently lacking in reports of network meta-analyses, even when published in journals with high impact factors.
Project description:INTRODUCTION:Children and young people placed in out-of-home care (OoHC) are often affected by a history of trauma and adverse childhood experiences. Trauma in early childhood can impact on children's health and psychosocial development, whereas early interventions can improve children's development and placement stability. Although several interventions and practice models have been developed to improve health and psychosocial outcomes for children and young people in OoHC, there remains a lack of rigorous research examining the impact of these interventions in OoHC settings, as there are no systematic reviews examining the impact these interventions and practice models have on the children and young people they serve. We aim to conduct a comprehensive systematic review to examine the effectiveness of interventions and practice models for improving health and psychosocial outcomes in children and young people living in OoHC and to identify relevant knowledge gaps. METHODS AND ANALYSIS:Major electronic databases including Medline, Medline in-process and other non-indexed citations, Embase, Cumulative Index to Nursing and Allied Health Literature, PsycInfo, Sociological Abstracts and all Evidence-Based Medicine Reviews incorporating: Cochrane Database of Systematic Reviews, American College of PhysiciansJournal Club, Database of Abstracts of Reviews of Effects,Cochrane Central Register of Controlled Trials, CochraneMethodology Register, Health Technology Assessment and National Health Service Economic Evaluation Database, will be systematically searched for any studies published between 2008 and 2018 of interventions and practice models developed to improve health and psychosocial outcomes for children and young people in OoHC. Two independent reviewers will assess titles and abstracts for eligibility according to prespecified selection criteria and will perform data extraction and quality appraisal. Meta-analyses and/or metaregression will be conducted where appropriate. ETHICS AND DISSEMINATION:This study will not collect primary data and formal ethical approval is therefore not required. Findings from this systematic review will be disseminated in a peer-reviewed publication and conference presentations. PROSPERO REGISTRATION NUMBER:CRD42019115082.
Project description:Authors and indexers are increasingly including terms for adverse drug effects in the titles, abstracts, or indexing of records in MEDLINE and Embase. However, it is not clear if this is the same for studies with nondrug adverse effects data. We therefore assessed the feasibility of using adverse effects terms when searching MEDLINE or Embase to retrieve papers of nondrug adverse effects. A collection of papers that reported data on nondrug adverse effects was sought from included studies of systematic reviews. Each included study was analysed to ascertain whether the corresponding record in MEDLINE and Embase included adverse effects terms in the title, abstract, or indexing. From 9129 records screened from DARE, 30 reviews evaluating nondrug adverse effects met our inclusion criteria. From these, 635 unique papers were included in our analysis. Sensitive searches for adverse effects required generic and specific named adverse effects terms using the title, abstract, and indexing. Records relating to surgical interventions were more likely to contain adverse effects terms than records relating to nonsurgical interventions. Using any adverse effects terms in the title, abstract or indexing in MEDLINE and Embase would have identified an average of 94% of papers on surgical adverse effect interventions per systematic review and 72% of papers on nonsurgical adverse effects per systematic review. Hence, while a generic nondrug adverse effect search filter may not yet be feasible, a filter for the adverse effects of surgical interventions may be within reach.
Project description:INTRODUCTION:Sexual dysfunction in men is common, and optimal treatment is complex. Although several systematic reviews concerning treatment approaches exist, a comprehensive overview without limitations concerning the population, interventions, or outcomes is lacking. AIM:To conduct a "review of reviews" to compare the effectiveness of pharmacologic, non-pharmacologic, and combined interventions. METHODS:9 electronic databases, relevant journals, and reference lists up to July 2018 were searched. For each intervention, only the most recent and comprehensive meta-analysis or systematic review was included. The methodologic quality of the reviews was appraised using the Assessment of Multiple Systematic Reviews-2 tool. MAIN OUTCOME MEASURE:Sexual functioning (via intravaginal ejaculatory latency time and international index of erectile function), sexual satisfaction, and adverse effects. RESULTS:30 systematic reviews were included. For premature ejaculation, several treatments, including oral pharmacotherapy (selective serotonin inhibitors, phosphodiesterase type 5 [PDE5] inhibitors, tricyclic antidepressants, and opioid analgesics), topical anesthetics, and combined drug and behavioral therapies demonstrated significant improvements of 1-5 minutes in the intravaginal ejaculatory latency time. Pharmacologic interventions (PDE5 inhibitors, penile injection, and testosterone), shockwave therapy, lifestyle modifications, and combined therapies (PDE5 inhibitors and psychological intervention) were effective in treating erectile dysfunction. Most pharmacologic therapies were associated with adverse effects. CONCLUSIONS:There is suggestive evidence that pharmacologic interventions or combined therapies are more effective than non-pharmacologic interventions for treating sexual dysfunction in men; however, a range of treatment options should be presented to individual patients so they may consider the risks and benefits of treatments differently. Evidence related to behavioral and psychological interventions is insufficient compared with that related to drug trials, highlighting the necessity for larger and better randomized controlled trials. Ciocanel O, Power K, Eriksen A. Interventions to Treat Erectile Dysfunction and Premature Ejaculation: An Overview of Systematic Reviews. Sex Med 2019;7:251-269.