Statistical approaches to harmonize data on cognitive measures in systematic reviews are rarely reported.
Ontology highlight
ABSTRACT:
SUBMITTER: Griffith LE
PROVIDER: S-EPMC4685455 | biostudies-literature | 2015 Feb
REPOSITORIES: biostudies-literature
Publications
Statistical approaches to harmonize data on cognitive measures in systematic reviews are rarely reported.
Griffith Lauren E LE van den Heuvel Edwin E Fortier Isabel I Sohel Nazmul N Hofer Scott M SM Payette Hélène H Wolfson Christina C Belleville Sylvie S Kenny Meghan M Doiron Dany D Raina Parminder P
Journal of clinical epidemiology 20141208 2
<h4>Objectives</h4>To identify statistical methods for harmonization, the procedures aimed at achieving the comparability of previously collected data, which could be used in the context of summary data and individual participant data meta-analysis of cognitive measures.<h4>Study design and setting</h4>Environmental scan methods were used to conduct two reviews to identify (1) studies that quantitatively combined data on cognition and (2) general literature on statistical methods for data harmon ...[more]
Similar Datasets
Project description:PURPOSE:The original COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist was developed to assess the methodological quality of single studies on measurement properties of Patient-Reported Outcome Measures (PROMs). Now it is our aim to adapt the COSMIN checklist and its four-point rating system into a version exclusively for use in systematic reviews of PROMs, aiming to assess risk of bias of studies on measurement properties. METHODS:For each standard (i.e., a design requirement or preferred statistical method), it was discussed within the COSMIN steering committee if and how it should be adapted. The adapted checklist was pilot-tested to strengthen content validity in a systematic review on the quality of PROMs for patients with hand osteoarthritis. RESULTS:Most important changes were the reordering of the measurement properties to be assessed in a systematic review of PROMs; the deletion of standards that concerned reporting issues and standards that not necessarily lead to biased results; the integration of standards on general requirements for studies on item response theory with standards for specific measurement properties; the recommendation to the review team to specify hypotheses for construct validity and responsiveness in advance, and subsequently the removal of the standards about formulating hypotheses; and the change in the labels of the four-point rating system. CONCLUSIONS:The COSMIN Risk of Bias checklist was developed exclusively for use in systematic reviews of PROMs to distinguish this application from other purposes of assessing the methodological quality of studies on measurement properties, such as guidance for designing or reporting a study on the measurement properties.
| S-EPMC5891552 | biostudies-literature
Project description:Background Subjective cognitive decline (SCD) is present in the early stage of preclinical Alzheimer’s disease (AD) and is associated with an increased risk of further cognitive decline and AD dementia later in life. Early detection of at-risk groups with subjective complaints is critical for targeted dementia prevention at the earliest. Accurate assessment of SCD is crucial. However, current measures lack important psychometric evaluations and or reporting. Objectives To systematically evaluate measurement properties of self-reported outcome measures (PROMs) used to assess SCD in the older adult population with or at risk of AD. Methods and analysis We used the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols 2015 Checklist for reporting. We conducted a literature search, screened, and included validation studies of SCD based on self-reported questionnaires from both population-based and clinical studies, conducted in older adults (≥ 55). We critically appraised the included primary studies using the Consensus-based Standards for the selection of health Measurement Instruments (COSMIN) guidelines. Results Sixteen studies met the inclusion criteria. The included studies reported psychometric properties of 17 SCD self-reported questionnaires. We extracted data on the structural validity, internal consistency, test-retest reliability, and cross-cultural validity and found a widespread proneness to bias across studies, and a marked heterogeneity is assessed and reported measurement properties that prevented the consolidation of results. Conclusion Our findings suggest that available SCD questionnaires lack content validity evaluation. Currently available measurements of SCD lack development and validation standards. Further work is needed to develop and validate SCD self-reported measurement with good quality measurement properties. Supplementary Information The online version contains supplementary material available at 10.1186/s13643-022-02018-y.
| S-EPMC9290248 | biostudies-literature
Project description:BackgroundResults of new studies should be interpreted in the context of what is already known to compare results and build the state of the science. This systematic review and meta-analysis aimed to identify and synthesise results from meta-research studies examining if original studies within health use systematic reviews to place their results in the context of earlier, similar studies.MethodsWe searched MEDLINE (OVID), EMBASE (OVID), and the Cochrane Methodology Register for meta-research studies reporting the use of systematic reviews to place results of original clinical studies in the context of existing studies. The primary outcome was the percentage of original studies included in the meta-research studies using systematic reviews or meta-analyses placing new results in the context of existing studies. Two reviewers independently performed screening and data extraction. Data were synthesised using narrative synthesis and a random-effects meta-analysis was performed to estimate the mean proportion of original studies placing their results in the context of earlier studies. The protocol was registered in Open Science Framework.ResultsWe included 15 meta-research studies, representing 1724 original studies. The mean percentage of original studies within these meta-research studies placing their results in the context of existing studies was 30.7% (95% CI [23.8%, 37.6%], I2=87.4%). Only one of the meta-research studies integrated results in a meta-analysis, while four integrated their results within a systematic review; the remaining cited or referred to a systematic review. The results of this systematic review are characterised by a high degree of heterogeneity and should be interpreted cautiously.ConclusionOur systematic review demonstrates a low rate of and great variability in using systematic reviews to place new results in the context of existing studies. On average, one third of the original studies contextualised their results. Improvement is still needed in researchers' use of prior research systematically and transparently-also known as the use of an evidence-based research approach, to contribute to the accumulation of new evidence on which future studies should be based.Systematic review registrationOpen Science registration number https://osf.io/8gkzu/.
| S-EPMC9446778 | biostudies-literature
Project description:BackgroundThere is increasing interest in using Patient Reported Outcome Measures (PROMs) within organisations delivering health related services. However, organisations have had mixed success in implementing PROMs and there is little understanding about why this may be. Thus, the purpose of this study was to identify the facilitators and barriers to implementing PROMs in organisations.MethodA systematic review of reviews was undertaken. Searches were conducted of five electronic databases: MEDLINE, EMBASE, CINAHL, PsycINFO and the Cochrane Database of Systematic Reviews, during the week of the 20th February 2017. Additional search methods included website searching and reference checking. To be included, a publication had to be a review of the literature, describe its methods and include information related to implementing PROMs. The reviews were extracted using a standardised form and assessed for their risk of bias using the Risk of Bias in Systematic Reviews tool. The findings were synthesised using the Consolidated Framework for Implementation Research. The protocol was registered on the International Prospective Register of Systematic Reviews database (PROSPERO) (CRD42017057491).ResultsInitially 2047 records were identified. After assessing eligibility, six reviews were included. These reviews varied in their review type and focus. Different issues arose at distinct stages of the implementation process. Organisations needed to invest time and resources in two key stages early in the implementation process: 'designing' the processes for using PROMs within an organisation; and 'preparing' an organisation and its staff. The 'designing' stage involved organisations planning not just which PROMs to use and how to administer them, but also how the data would be used for clinical purposes. The 'preparing' stage involved getting an organisation and its staff ready to use PROMs, particularly persuading clinicians of the validity and value of PROMs, delivering training, and developing electronic systems. Having an implementation lead overseeing the process and developing the process based on feedback were also identified as facilitating implementation.ConclusionOrganisations implementing PROMs need to invest time and resources in 'designing' the PROMs strategy and 'preparing' the organisation to use PROMs. Focusing on these earlier stages may prevent problems arising when PROMs are used in practice.
| S-EPMC6170512 | biostudies-literature
Project description:OBJECTIVES:Issues arising from a lack of outcome standardisation in health research may be addressed by the use of core outcome sets (COS), which represent agreed-upon recommendations regarding what outcomes should be measured as a minimum in studies of a health condition. This review investigated the scope, outcomes, and development methods of consensus-based COS for cancer, and their approaches and criteria for selecting instruments to assess core patient-reported outcomes (PROs). METHODS:Studies that used a consensus-driven approach to develop a COS containing PROs, for use in research with cancer populations, were sought via MEDLINE, CINAHL, Embase, Cochrane Library, and grey literature. RESULTS:Seventeen studies met the inclusion criteria. Most COS (82%) were specific to a cancer type (prostate, esophageal, head and neck, pancreatic, breast, ovarian, lung, or colorectal) and not specific to an intervention or treatment (76%). Conducting a systematic review was the most common approach to identifying outcomes (88%) and administering a Delphi survey was the most common approach to prioritising outcomes (71%). The included COS contained 90 PROs, of which the most common were physical function, sexual (dys) function, pain, fatigue, and emotional function. Most studies (59%) did not address how to assess the core PROs included in a set, while 7 studies (41%) recommended specific instruments. Their approaches to instrument appraisal and selection varied. CONCLUSION:Efforts to standardise outcome assessment via the development of COS may be undermined by a lack of recommendations on how to measure core PROs. To optimise COS usefulness and adoption, valid and reliable instruments for the assessment of core PROs should be recommended with the aid of resources designed to facilitate this process.
| S-EPMC7492323 | biostudies-literature
Project description:INTRODUCTION:Meta-analyses of clinical trials often provide sufficient information for decision-makers to evaluate whether chance can explain apparent differences between interventions. Interpretation of the magnitude and importance of treatment effects beyond statistical significance can, however, be challenging, particularly for patient-reported outcomes (PROs) measured using questionnaires with which clinicians have limited familiarity. The objectives of our study are to systematically evaluate Cochrane systematic review authors' approaches to calculation, reporting and interpretation of pooled estimates of patient-reported outcome measures (PROMs) in meta-analyses. METHODS AND ANALYSIS:We will conduct a methodological survey of a random sample of Cochrane systematic reviews published from 1 January 2015 to 1 April 2017 that report at least one statistically significant pooled result for at least one PRO in the abstract. Author pairs will independently review all titles, abstracts and full texts identified by the literature search, and they will extract data using a standardised data extraction form. We will extract the following: year of publication, number of included trials, number of included participants, clinical area, type of intervention(s) and control(s), type of meta-analysis and use of the Grading of Recommendations, Assessment, Development and Evaluation approach to rate the quality of evidence, as well as information regarding the characteristics of PROMs, calculation and presentation of PROM effect estimates and interpretation of PROM effect estimates. We will document and summarise the methods used for the analysis, reporting and interpretation of each summary effect measure. We will summarise categorical variables with frequencies and percentages and continuous outcomes as means and/or medians and associated measures of dispersion. ETHICS AND DISSEMINATION:Ethics approval for this study is not required. We will disseminate the results of this review in peer-reviewed publications and conference presentations.
| S-EPMC5623559 | biostudies-literature
Project description:BackgroundUnderstanding which patient-reported outcome measures are being collected and utilized in clinical practice and research for patients with neck pain will help to inform recommendations for a core set of measures that provide value to patients and clinicians during diagnosis, clinical decision-making, goal setting and evaluation of responsiveness to treatment. Therefore, the aim of this study was to conduct a review of systematic reviews using a qualitative synthesis on the use of patient-reported outcome measures (PROMs) for patients presenting with neck pain to physical therapy.MethodsAn electronic search of systematic reviews and guideline publications was performed using MEDLINE (OVID), Embase (Elsevier), CINAHL Complete (EBSCOhost), and Web of Science (Clarivate) databases to identify reviews that evaluated physical therapy interventions or interventions commonly performed by a physical therapist for individuals with neck pain and included at least one patient-reported outcome measure. The frequency and variability in which the outcome measures were reported among the studies in the review and the constructs for which they measured were evaluated. The evaluation of a core set of outcome measures was assessed. Risk of bias and quality assessment was performed using A Measurement Tool to Assess systematic Reviews 2.ResultsOf the initial 7,003 articles, a total of 37 studies were included in the final review. Thirty-one PROMs were represented within the 37 reviews with eleven patient-reported outcome measures in three or more reviews. The eleven PROMs assessed the constructs of disability, pain intensity, psychosocial factors and quality of life. The greatest variability was found amongst individual measures assessing psychosocial factors. Assessment of psychosocial factors was the least represented construct in the included studies. Overall, the most frequently utilized patient reported outcome measures were the Neck Disability Index, Visual Analog Scale, and Numeric Pain Rating Scale. The most frequently used measures evaluating the constructs of disability, pain intensity, quality of life and psychosocial functioning included the Neck Disability Index, Visual Analog Scale, Short-Form-36 health survey and Fear Avoidance Belief Questionnaire respectively. Overall risk of bias and quality assessment confidence levels ranged from critically low (2 studies), low (12 studies), moderate (8 studies), and high (15 studies).ConclusionThis study identified a core set of patient-reported outcome measures that represented the constructs of disability, pain intensity and quality of life. This review recommends the collection and use of the Neck Disability Index and the Numeric Pain Rating Scale or Visual Analog Scale. Recommendation for a QoL measure needs to be considered in the context of available resources and administrative burden. Further research is needed to confidently recommend a QoL and psychosocial measure for patients presenting with neck pain. Other measures that were not included in this review but should be further evaluated for patients with neck pain are the Patient Reported Outcomes Measurement Information System (PROMIS) Physical function, PROMIS Pain Interference and the Optimal Screening for Prediction of Referral and Outcome Yellow Flag (OSPRO-YF) tool.
| S-EPMC10545655 | biostudies-literature
Project description:BackgroundSystematic reviews (SRs) can provide accurate and reliable evidence, typically about the effectiveness of health interventions. Evidence is dynamic, and if SRs are out-of-date this information may not be useful; it may even be harmful. This study aimed to compare five statistical methods to identify out-of-date SRs.MethodsA retrospective cohort of SRs registered in the Cochrane Pregnancy and Childbirth Group (CPCG), published between 2008 and 2010, were considered for inclusion. For each eligible CPCG review, data were extracted and "3-years previous" meta-analyses were assessed for the need to update, given the data from the most recent 3 years. Each of the five statistical methods was used, with random effects analyses throughout the study.ResultsEighty reviews were included in this study; most were in the area of induction of labour. The numbers of reviews identified as being out-of-date using the Ottawa, recursive cumulative meta-analysis (CMA), and Barrowman methods were 34, 7, and 7 respectively. No reviews were identified as being out-of-date using the simulation-based power method, or the CMA for sufficiency and stability method. The overall agreement among the three discriminating statistical methods was slight (Kappa = 0.14; 95% CI 0.05 to 0.23). The recursive cumulative meta-analysis, Ottawa, and Barrowman methods were practical according to the study criteria.ConclusionOur study shows that three practical statistical methods could be applied to examine the need to update SRs.
| S-EPMC3502410 | biostudies-literature
Project description:BackgroundThere are a variety of ways in which accuracy of clinical tests can be summarised in systematic reviews. Variation in reporting of summary measures has only been assessed in a small survey restricted to meta-analyses of screening studies found in a single database. Therefore, we performed this study to assess the measures of accuracy used for reporting results of primary studies as well as their meta-analysis in systematic reviews of test accuracy studies.MethodsRelevant reviews on test accuracy were selected from the Database of Abstracts of Reviews of Effectiveness (1994-2000), which electronically searches seven bibliographic databases and manually searches key resources. The structured abstracts of these reviews were screened and information on accuracy measures was extracted from the full texts of 90 relevant reviews, 60 of which used meta-analysis.ResultsSensitivity or specificity was used for reporting the results of primary studies in 65/90 (72%) reviews, predictive values in 26/90 (28%), and likelihood ratios in 20/90 (22%). For meta-analysis, pooled sensitivity or specificity was used in 35/60 (58%) reviews, pooled predictive values in 11/60 (18%), pooled likelihood ratios in 13/60 (22%), and pooled diagnostic odds ratio in 5/60 (8%). Summary ROC was used in 44/60 (73%) of the meta-analyses. There were no significant differences in measures of test accuracy among reviews published earlier (1994-97) and those published later (1998-2000).ConclusionsThere is considerable variation in ways of reporting and summarising results of test accuracy studies in systematic reviews. There is a need for consensus about the best ways of reporting results of test accuracy studies in reviews.
| S-EPMC100326 | biostudies-literature
Project description:BackgroundSystematic reviews with meta-analyses often contain many statistical tests. This multiplicity may increase the risk of type I error. Few attempts have been made to address the problem of statistical multiplicity in systematic reviews. Before the implications are properly considered, the size of the issue deserves clarification. Because of the emphasis on bias evaluation and because of the editorial processes involved, Cochrane reviews may contain more multiplicity than their non-Cochrane counterparts. This study measured the quantity of statistical multiplicity present in a population of systematic reviews and aimed to assess whether this quantity is different in Cochrane and non-Cochrane reviews.Methods/principal findingsWe selected all the systematic reviews published by the Cochrane Anaesthesia Review Group containing a meta-analysis and matched them with comparable non-Cochrane reviews. We counted the number of statistical tests done in each systematic review. The median number of tests overall was 10 (interquartile range (IQR) 6 to 18). The median was 12 in Cochrane and 8 in non-Cochrane reviews (difference in medians 4 (95% confidence interval (CI) 2.0-19.0). The proportion that used an assessment of risk of bias as a reason for doing extra analyses was 42% in Cochrane and 28% in non-Cochrane reviews (difference in proportions 14% (95% CI -8 to 36). The issue of multiplicity was addressed in 6% of all the reviews.Conclusion/significanceStatistical multiplicity in systematic reviews requires attention. We found more multiplicity in Cochrane reviews than in non-Cochrane reviews. Many of the reasons for the increase in multiplicity may well represent improved methodological approaches and greater transparency, but multiplicity may also cause an increased risk of spurious conclusions. Few systematic reviews, whether Cochrane or non-Cochrane, address the issue of multiplicity.
| S-EPMC3229598 | biostudies-literature