Dataset Information

Improving the quality of toxicology and environmental health systematic reviews: What journal editors can do.

ABSTRACT: Systematic reviews are fast increasing in prevalence in the toxicology and environmental health literature. However, how well these complex research projects are being conducted and reported is unclear. Since editors have an essential role in ensuring the scientific quality of manuscripts being published in their journals, a workshop was convened where editors, systematic review practitioners, and research quality control experts could discuss what editors can do to ensure the systematic reviews they publish are of sufficient scientific quality. Interventions were explored along four themes: setting standards; reviewing protocols; optimizing editorial workflows; and measuring the effectiveness of editorial interventions. In total, 58 editorial interventions were proposed. Of these, 26 were shortlisted for being potentially effective, and 5 were prioritized as short-term actions that editors could relatively easily take to improve the quality of published systematic reviews. Recent progress in improving systematic reviews is summarized, and outstanding challenges to further progress are highlighted.

SUBMITTER: Whaley P

PROVIDER: S-EPMC9472299 | biostudies-literature | 2021

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Improving the quality of toxicology and environmental health systematic reviews: What journal editors can do.

Whaley Paul P Blaauboer Bas J BJ Brozek Jan J Cohen Hubal Elaine A EA Hair Kaitlyn K Kacew Sam S Knudsen Thomas B TB Kwiatkowski Carol F CF Mellor David T DT Olshan Andrew F AF Page Matthew J MJ Rooney Andrew A AA Radke Elizabeth G EG Shamseer Larissa L Tsaioun Katya K Tugwell Peter P Wikoff Daniele D Woodruff Tracey J TJ

ALTEX 20210622 3

Systematic reviews are fast increasing in prevalence in the toxicology and environmental health literature. However, how well these complex research projects are being conducted and reported is unclear. Since editors have an essential role in ensuring the scientific quality of manuscripts being published in their journals, a workshop was convened where editors, systematic review practitioners, and research quality control experts could discuss what editors can do to ensure the systematic reviews ...[more]

PMID: 34164697

Similar Datasets

Project description:BackgroundSynthesizing environmental health science is crucial to taking action to protect public health. Procedures for evidence evaluation and integration are transitioning from "expert-based narrative" to "systematic" review methods. However, little is known about the methodology being utilized for either type of review.ObjectivesTo appraise the methodological strengths and weaknesses of a sample of "expert-based narrative" and "systematic" reviews in environmental health.MethodsWe conducted a comprehensive search of multiple databases and identified relevant reviews using pre-specified eligibility criteria. We applied a modified version of the Literature Review Appraisal Toolkit (LRAT) to three environmental health topics that assessed the utility, validity and transparency of reviews.ResultsWe identified 29 reviews published between 2003 and 2019, of which 13 (45%) were self-identified as systematic reviews. Across every LRAT domain, systematic reviews received a higher percentage of "satisfactory" ratings compared to non-systematic reviews. In eight of these domains, there was a statistically significant difference observed between the two types of reviews and "satisfactory" ratings. Non-systematic reviews performed poorly with the majority receiving an "unsatisfactory" or "unclear" rating in 11 of the 12 domains. Systematic reviews performed poorly in six of the 12 domains; 10 (77%) did not state the reviews objectives or develop a protocol; eight (62%) did not state the roles and contribution of the authors, or evaluate the internal validity of the included evidence consistently using a valid method; and only seven (54%) stated a pre-defined definition of the evidence bar on which their conclusions were based, or had an author disclosure of interest statement.DiscussionSystematic reviews produced more useful, valid, and transparent conclusions compared to non-systematic reviews, but poorly conducted systematic reviews were prevalent. Ongoing development and implementation of empirically based systematic review methods are required in environmental health to ensure transparent and timely decision making to protect the public's health.

Project description:BackgroundSystematic reviews are important for informing clinical practice and health policy. The aim of this study was to examine the bibliometrics of systematic reviews and to determine the amount of variance in citations predicted by the journal impact factor (JIF) alone and combined with several other characteristics.MethodsWe conducted a bibliometric analysis of 1,261 systematic reviews published in 2008 and the citations to them in the Scopus database from 2008 to June 2012. Potential predictors of the citation impact of the reviews were examined using descriptive, univariate and multiple regression analysis.ResultsThe mean number of citations per review over four years was 26.5 (SD ± 29.9) or 6.6 citations per review per year. The mean JIF of the journals in which the reviews were published was 4.3 (SD ± 4.2). We found that 17% of the reviews accounted for 50% of the total citations and 1.6% of the reviews were not cited. The number of authors was correlated with the number of citations (r = 0.215, P < 0.001). Higher numbers of citations were associated with the following characteristics: first author from the United States (36.5 citations), an ICD-10 chapter heading of Neoplasms (31.8 citations), type of intervention classified as Investigation, Diagnostics or Screening (34.7 citations) and having an international collaboration (32.1 citations). The JIF alone explained more than half of the variation in citations (R(2) = 0.59) in univariate analysis. Adjusting for both JIF and type of intervention increased the R2 value to 0.81. Fourteen percent of reviews published in the top quartile of JIFs (? 5.16) received citations in the bottom quartile (eight or fewer), whereas 9% of reviews published in the lowest JIF quartile (? 2.06) received citations in the top quartile (34 or more). Six percent of reviews in journals with no JIF were also in the first quartile of citations.ConclusionsThe JIF predicted over half of the variation in citations to the systematic reviews. However, the distribution of citations was markedly skewed. Some reviews in journals with low JIFs were well-cited and others in higher JIF journals received relatively few citations; hence the JIF did not accurately represent the number of citations to individual systematic reviews.

Project description:PurposeSystematic reviews evaluating and comparing the measurement properties of outcome measurement instruments (OMIs) play an important role in OMI selection. Earlier overviews of review quality (2007, 2014) evidenced substantial concerns with regards to alignment to scientific standards. This overview aimed to investigate whether the quality of recent systematic reviews of OMIs lives up to the current scientific standards.MethodsOne hundred systematic reviews of OMIs published from June 1, 2021 onwards were randomly selected through a systematic literature search performed on March 17, 2022 in MEDLINE and EMBASE. The quality of systematic reviews was appraised by two independent reviewers. An updated data extraction form was informed by the earlier studies, and results were compared to these earlier studies' findings.ResultsA quarter of the reviews had an unclear research question or aim, and in 22% of the reviews the search strategy did not match the aim. Half of the reviews had an incomprehensive search strategy, because relevant search terms were not included. In 63% of the reviews (compared to 41% in 2014 and 30% in 2007) a risk of bias assessment was conducted. In 73% of the reviews (some) measurement properties were evaluated (58% in 2014 and 55% in 2007). In 60% of the reviews the data were (partly) synthesized (42% in 2014 and 7% in 2007); evaluation of measurement properties and data syntheses was not conducted separately for subscales in the majority. Certainty assessments of the quality of the total body of evidence were conducted in only 33% of reviews (not assessed in 2014 and 2007). The majority (58%) did not make any recommendations on which OMI (not) to use.ConclusionDespite clear improvements in risk of bias assessments, measurement property evaluation and data synthesis, specifying the research question, conducting the search strategy and performing a certainty assessment remain poor. To ensure that systematic reviews of OMIs meet current scientific standards, more consistent conduct and reporting of systematic reviews of OMIs is needed.

Dataset Information

Improving the quality of toxicology and environmental health systematic reviews: What journal editors can do.

Publications

Improving the quality of toxicology and environmental health systematic reviews: What journal editors can do.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets