Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist.
Ontology highlight
ABSTRACT:
SUBMITTER: Moons KG
PROVIDER: S-EPMC4196729 | biostudies-literature | 2014 Oct
REPOSITORIES: biostudies-literature
Publications
Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist.
Moons Karel G M KG de Groot Joris A H JA Bouwmeester Walter W Vergouwe Yvonne Y Mallett Susan S Altman Douglas G DG Reitsma Johannes B JB Collins Gary S GS
PLoS medicine 20141014 10
Carl Moons and colleagues provide a checklist and background explanation for critically appraising and extracting data from systematic reviews of prognostic and diagnostic prediction modelling studies. Please see later in the article for the Editors' Summary. ...[more]
Similar Datasets
Project description:In this letter, we briefly describe how we selected and implemented the quality criteria checklist (QCC) as a critical appraisal tool in rapid systematic reviews conducted to inform public health advice, guidance and policy during the COVID-19 pandemic. As these rapid reviews usually included a range of study designs, it was key to identify a single tool that would allow for reliable critical appraisal across most experimental and observational study designs and applicable to a range of topics. After carefully considering a number of existing tools, the QCC was selected as it had good interrater agreement between three reviewers (Fleiss kappa coefficient 0.639) and was found to be easy and fast to apply once familiar with the tool. The QCC consists of 10 questions, with sub-questions to specify how it should be applied to a specific study design. Four of these questions are considered as critical (on selection bias, group comparability, intervention/exposure assessment and outcome assessment) and the rating of a study (high, moderate or low methodological quality) depends on the responses to these four critical questions. Our results suggest that the QCC is an appropriate critical appraisal tool to assess experimental and observational studies within COVID-19 rapid reviews. This study was done at pace during the COVID-19 pandemic; further reliability analyses should be conducted, and more research is needed to validate the QCC across a range of public health topics.
| S-EPMC10042397 | biostudies-literature
Project description:ObjectivesTo develop a consensus-based checklist that can be used as a minimum standard to appraise the comprehensiveness, transparency and consistency of cost-of-illness (COI) studies. This is important when, for instance, reviewing and assessing COI studies as part of a systematic review or when building an economic model.MethodsThe development process of the consensus-based checklist involved six steps: (i) a scoping review, (ii) an assessment and comparison of the different checklists and their questions, (iii) the development of a (preliminary) checklist, (iv) expert interviews, (v) the finalization of the checklist, and (vi) the development of guidance statements explaining each question.ResultsThe result was a consensus-based checklist for the critical appraisal of COI studies, comprising seventeen main questions (and some additional subquestions) across three domains: (i) study characteristics; (ii) methodology and cost analysis; and (iii) results and reporting. Guidance statements were developed describing the purpose and meaning behind each question and listing examples of best practice. The following answer categories were suggested to be applied when answering the questions in the checklist: Yes, Partially, No, Not Applicable, or Unclear.ConclusionsThe consensus-based checklist for COI studies is a first step toward standardizing the critical appraisal of COI studies and is one that could be considered a minimum standard. The checklist can help to improve comprehensiveness, transparency and consistency in COI studies, to address heterogeneity, and to enable better comparability of methodological approaches across international studies.
| S-EPMC11574538 | biostudies-literature
Project description:BackgroundPrognosis research refers to the investigation of association between a baseline health state, patient characteristic and future outcomes. The findings of several prognostic studies can be summarized in systematic reviews (SRs), but some characteristics of prognostic studies may result in difficulties when performing the analyses. This study aimed to investigate trends in the volume and quality of SRs of prognostic studies in the literature.MethodsWe conducted a systematic review in five high-impact clinical journals (Annals of Internal Medicine, BMJ, Circulation, JAMA, and Stroke) to identify SRs of prognosis studies focused on fundamental prognosis research and prognostic factor research published between 2000 and 2012. We excluded studies of clinical prediction guides or implementation studies. The quality of the SRs was rated based on the Meta-analysis of Observational Studies in Epidemiology (MOOSE) and the PRISMA checklists.ResultsOver the 13-year period, 1065 SRs were published. Of these, 198 were SRs of prognosis studies. The proportion of all SRs to published articles increased from 0.86% in 2000 to 4.2% in 2012. Likewise, the proportion of prognosis SRs to all SRs increased from 10.3% in 2000 to 17.7% in 2012. MOOSE and PRISMA mean summary scores consistently increased over time for all journals, indicating that the quality of reporting in these SRs has steadily improved. However, several items were not consistently well reported by investigators.ConclusionsThis study shows that there is a growing number of SRs of prognosis studies. However, the quality is suboptimal when assessed with the generic reporting guidelines for observational studies. New reporting guidelines and risk of bias tools for prognosis studies are needed to improve the quality of future research in this field.
| S-EPMC6460771 | biostudies-literature
Project description:BackgroundSystematic reviews of studies of clinical prediction models are becoming increasingly abundant in the literature. Data extraction and risk of bias assessment are critical steps in any systematic review. CHARMS and PROBAST are the standard tools used for these steps in these reviews of clinical prediction models.ResultsWe developed an Excel template for data extraction and risk of bias assessment of clinical prediction models including both recommended tools. The template makes it easier for reviewers to extract data, to assess the risk of bias and applicability, and to produce results tables and figures ready for publication.ConclusionWe hope this template will simplify and standardize the process of conducting a systematic review of prediction models, and promote a better and more comprehensive reporting of these systematic reviews.
| S-EPMC9936746 | biostudies-literature
Project description:ObjectivesThis study explored experts' views on the development of a proposed checklist for cost-of-illness (COI) studies. It also investigated experts' perspectives on the use of COI studies and quality/critical appraisal tools used for COI studies as well as their experiences with the use of these tools.MethodsSemi-structured, open-ended interviews were conducted with health economists and other experts working with COI studies and with experience of developing health economic guidelines or checklists. Participants were selected purposively using network and snowball sampling. A framework approach was applied for the thematic data analysis. Findings were reported narratively.ResultsTwenty-one experts from eleven different countries were interviewed. COI studies were found to be relevant to estimate the overall burden of a disease, to draw attention to disease areas, to understand different cost components, to explain cost variability, to inform decision making, and to provide input for full economic evaluations. Experts reported a lack of a standardized critical appraisal tool for COI studies. Their experience related predominantly to guidelines and checklists designed for full economic evaluations to review and assess COI studies. The following themes emerged when discussing the checklist: (i) the need for a critical appraisal tool, (ii) format and practicality, (iii) assessing the questions, (iv) addressing subjectivity, and (v) guidance requirements.ConclusionsThe interviews provided relevant input for the development of a checklist for COI studies that could be used as a minimum standard and for international application. The interviews confirmed the important need for a checklist for the critical appraisal of COI studies.
| S-EPMC11574535 | biostudies-literature
Project description:BackgroundThe quality of nutrition-related systematic reviews (SRs) is an unstudied but important factor affecting their usefulness.ObjectivesThe objectives were to evaluate the reporting quality of published SRs and to identify areas of improvement.DesignDescriptive and exploratory analyses of the reporting quality (7 nutrition items and 28 SR reporting items) of all English-language SRs published through July 2007 linking micronutrients and health outcomes in humans were conducted. Factors that may be associated with reporting quality were also evaluated.ResultsWe identified 141 eligible SRs of 21 micronutrients. Ninety SRs that included only interventional studies met a higher proportion of our reporting criteria (median: 62%; interquartile range: 51%, 72%) than did 31 SRs with only observational studies (median: 53%; interquartile range: 47%, 60%) or 20 SRs with both study designs (median: 47%; interquartile range: 39%, 52%) (P < 0.001). SRs published after consensus reporting standards (since 2003) met a higher proportion of the reporting criteria than did earlier SRs (median: 59% compared with 50%; P = 0.01); however, the reporting of nutrition variables remained unchanged (median: 38% compared with 33%; P = 0.7). The least-reported nutrition criteria were baseline nutrient exposures (28%) and effects of measurement errors from nutrition exposures (24%). Only 58 SRs (41%) used quality scales or checklists to assess the methodologic quality of the primary studies included.ConclusionsThe reporting quality of SRs has improved 3 y after publication of SR reporting standards, but the reporting of nutrition variables has not. Improved adherence to consensus methods and reporting standards should improve the utility of nutrition SRs.
| S-EPMC2667458 | biostudies-literature
Project description:BackgroundCancer-related anorexia remains one of the most prevalent and troublesome clinical problems experienced by patients with cancer during and after therapy. To ensure high-quality care, systematic reviews (SRs) are seen as the best guide. Considering the methodology quality of SRs varies, we undertook a comprehensive overview, and critical appraisal of pertinent SRs.MethodsEight databases (between the inception of each database and September 1, 2017) were searched for SRs on the management of cancer-related anorexia. Two researchers evaluated the methodological quality of each SR by using the Revised Assessment of Multiple Systematic Reviews (R-AMSTAR) checklist. Characteristics of the "high quality" SRs were abstracted, included information on relevant studies numbers, study design, population, intervention, control, outcome and result.ResultsEighteen SRs met the inclusion criteria. The R-AMSTAR scores of methodological quality ranged from 18 to 41 out of 44, with an average score of 30. Totally eight SRs scored ≥31 points, which showed high methodological quality, and would be used for data extraction to make summaries. Anamorelin had some positive effects to relieve cancer anorexia-cachexia syndrome (CACS) and improve the quality of life (QoL). Megestrol Acetate (MA) could improve appetite, and was associated with slight weight gain for CACS. Oral nutritional interventions were effective in increasing nutritional intake and improving some aspects of QoL in patients with cancer who were malnourished or at nutritional risk. The use of thalidomide, Eicosapentaenoic Acid, and minerals, vitamins, proteins, or other supplements for the treatment of cachexia in cancer were uncertain, and there was inadequate evidence to recommend it to clinical practices, the same situation in Chinese Herb Medicine and acupuncture (acupuncture and related therapies were effective in improving QoL) for treating anorexia in cancer patients, warranting further RCTs in these areas.ConclusionsAnamorelin, MA, oral nutrition interventions, and acupuncture could be considered to be applied in patients with cancer-related anorexia. Future RCTs and SRs with high quality on the pharmaceutical or non-pharmaceutical interventions of anorexia in cancer patients are warranted.
| S-EPMC6085669 | biostudies-literature
Project description: Not available
| S-EPMC5833365 | biostudies-literature
Project description:Data extraction is one of the most time-consuming tasks in performing a systematic review. Extraction is often onto some sort of form. Sharing completed forms can be used to check quality and accuracy of extraction or for re-cycling data to other researchers for updating. However, validating each piece of extracted data is time-consuming and linking to source problematic.In this methodology paper, we summarize three methods for reporting the location of data in original full-text reports, comparing their advantages and disadvantages.
| S-EPMC5544999 | biostudies-other
Project description:BackgroundPeople presenting with first-episode psychosis (FEP) have heterogenous outcomes. More than 40% fail to achieve symptomatic remission. Accurate prediction of individual outcome in FEP could facilitate early intervention to change the clinical trajectory and improve prognosis.AimsWe aim to systematically review evidence for prediction models developed for predicting poor outcome in FEP.MethodA protocol for this study was published on the International Prospective Register of Systematic Reviews, registration number CRD42019156897. Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidance, we systematically searched six databases from inception to 28 January 2021. We used the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies and the Prediction Model Risk of Bias Assessment Tool to extract and appraise the outcome prediction models. We considered study characteristics, methodology and model performance.ResultsThirteen studies reporting 31 prediction models across a range of clinical outcomes met criteria for inclusion. Eleven studies used logistic regression with clinical and sociodemographic predictor variables. Just two studies were found to be at low risk of bias. Methodological limitations identified included a lack of appropriate validation, small sample sizes, poor handling of missing data and inadequate reporting of calibration and discrimination measures. To date, no model has been applied to clinical practice.ConclusionsFuture prediction studies in psychosis should prioritise methodological rigour and external validation in larger samples. The potential for prediction modelling in FEP is yet to be realised.
| S-EPMC7612705 | biostudies-literature