Dataset Information

Does it work? Using a Meta-Impact score to examine global effects in quasi-experimental intervention studies.

ABSTRACT: The evaluation of applied psychological interventions in the workplace or elsewhere is challenging. Randomisation and matching are difficult to achieve and this often results in substantial heterogeneity within intervention and control groups. As a result, traditional comparison of group means using null hypothesis significance testing may mask effects experienced by some participants. Using longitudinal studies of coaching interventions designed to provide support for dyslexic employees, this study describes and evaluates a different approach using a Meta-Impact score. We offer a conceptual rationale for our method, illustrate how this score is calculated and analysed, and show how it highlights person-specific variations in how participants react and respond to interventions. We argue that Meta-Impact is an incremental supplement to traditional variable-centric group-wise comparisons and can more accurately demonstrate in practice the extent to which an intervention worked. Such methods are needed for applied research, where personalized intervention protocols may require impact analysis for policy, legal and ethical purposes, despite modest sample sizes.

SUBMITTER: Doyle NE

PROVIDER: S-EPMC8929616 | biostudies-literature | 2022

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Does it work? Using a Meta-Impact score to examine global effects in quasi-experimental intervention studies.

Doyle Nancy Elizabeth NE McDowall Almuth A Randall Raymond R Knight Kate K

PloS one 20220317 3

The evaluation of applied psychological interventions in the workplace or elsewhere is challenging. Randomisation and matching are difficult to achieve and this often results in substantial heterogeneity within intervention and control groups. As a result, traditional comparison of group means using null hypothesis significance testing may mask effects experienced by some participants. Using longitudinal studies of coaching interventions designed to provide support for dyslexic employees, this s ...[more]

PMID: 35298519

Similar Datasets

Project description:BackgroundThe magnitude of the benefit associated with screening has been debated. We present a meta-analysis of quasi-experimental studies on the effects of mammography screening.MethodsWe searched MEDLINE/PubMed and Embase for articles published through January 31, 2013. Studies were included if they reported: 1) a population-wide breast cancer screening program using mammography with 5+ years of data post-implementation; 2) a comparison group with equal access to therapies; and 3) breast cancer mortality. Studies excluded were: RCTs, case-control, or simulation studies. We defined quasi-experimental as studies that compared either geographical, historical or birth cohorts with a screening program to an equivalent cohort without a screening program. Meta-analyses were conducted in Stata using the metan command, random effects. Meta-analyses were conducted separately for ages screened: under 50, 50 to 69 and over 70 and weighted by population and person-years.ResultsAmong 4,903 published papers that were retrieved, 19 studies matched eligibility criteria. Birth cohort studies reported a significant benefit for women screened <age 50, but not for women screened ages 50-69. Significant reductions in breast cancer mortality were observed in historical comparisons. For geographical comparisons, there was a significant 20% reduction in mortality for women <age 50 and a significant 21-22% reduction for women ages 50-69. Studies that tested the interaction of geographical and historical comparisons produced a pooled, significant 13-17% reduction in incident breast cancer mortality for women ages 50-69, but the effects in most individual studies were non-significant. All studies of women ages 70+ were non-significant.ConclusionsMammography screening may have modest effects on cancer mortality between the ages of 50 and 69 and non-significant effects for women older than age 70. Results are consistent with meta-analyses of RCTs. Effects on total mortality could not be assessed because of the limited number of studies.

Project description:BACKGROUND:From 2000 to 2008, in urban areas in Spain, adolescent fertility and abortion rates underwent unprecedented increases, consecutive to intensive immigration from developing countries. To address unmet needs for contraception information and services, a community-based, gender-sensitive and culturally adapted brief counselling intervention (SIRIAN program) was launched in some deprived neighbourhoods with a high proportion of immigrants in Barcelona. Once a randomized controlled trial demonstrated its effectiveness in increasing the use of contraceptives, we aim to examine its population impact on adolescent fertility rates. METHODS:Quasi-experimental study with comparison group, using population data from 2005 to 2016. Five neighbourhoods in the lowest tercile of Disposable Household Income were intervened in 2011-13. The comparison group included the three neighbourhoods which were in the same municipal district and in the lowest Disposable Household Income tercile, and displayed the highest adolescent fertility rates. Generalized linear models were fitted to assess absolute adolescent fertility rates and adjusted by immigrant population between pre-intervention (2005-10) and post-intervention periods (2011-16); Difference in Differences and relative pre-post changes analysis were performed. RESULTS:In 2005-10 the intervention group adolescent fertility rate was 27.90 (per 1000 women 15-19) and 21.84 in the comparison group. In 2011-16 intervention areas experienced great declines (adolescent fertility rate change: -?12.30 (-?12.45 to -?12.21); p?<?0.001), while comparison neighbourhoods remained unchanged (adolescent fertility rate change: 1.91 (-?2.25 to 6.07); p?=?0.368). A reduction of -?10.97 points (-?13.91 to -?8.03); p?<?0.001) is associated to the intervention. CONCLUSION:Adolescent fertility rate significantly declined in the intervention group but remained stable in the comparison group. This quasi-experimental study provide evidence that, in a country with universal health coverage, a community counselling intervention that increases access to contraception, knowledge and sexual health care in hard-to-reach segments of the population can contribute to substantially reduce adolescent fertility rates. Reducing adolescent fertility rates could become a feasible goal in cities with similar conditions.

Project description:Aims and objectivesThe aim of this quasi-experimental longitudinal intervention study was to investigate new graduate nurses' professional competence development after preceptors' participation in an education intervention.BackgroundNew graduate registered nurses are expected to be competent in many areas of nursing. Expectations that are sometimes unrealistic may cause a sense of inadequacy and stress, and this may in turn prevent them from fully deploying their competencies. Competence development is related to practice environment, occupational commitment, empowerment, and work experience. Orientation or transition programs have been designed to ensure new graduate nurses' competence, and preceptors and preceptorship could also have significant influence on their competence development.DesignA quasi-experimental longitudinal intervention study.MethodsThe data was collected from October 2015 to November 2017. Participating wards were randomized into intervention and control groups. The intervention group preceptors had an eight-hour education intervention that focused on new employees' orientation, particularly from new graduates' point of view. Wards in the control group continued to precept as before. The Nurse Competence Scale was used for new graduates' self-assessment at baseline and at three-month and nine-month follow-up. This study is reported in accordance with the TREND Statement Checklist.ResultsThe education intervention aimed at preceptors did not have impact on the intervention group NGRNs' competence development. There were no statistically significant differences between the groups and effect size remained small.ConclusionsThe preceptors' education intervention was not effective enough to develop new graduates' professional competence so that it would have differed from that of the graduates receiving conventional orientation at the university hospital. This study confirmed that competence development is a complex and multidimensional phenomenon and organizations should invest in new graduate registered nurses' competence development during their early career. Preceptors' education and development of preceptorship and transition programs are an important part of overall competence development in complex health care environments.Trial registrationRetrospectively registered.

Dataset Information

Does it work? Using a Meta-Impact score to examine global effects in quasi-experimental intervention studies.

Publications

Does it work? Using a Meta-Impact score to examine global effects in quasi-experimental intervention studies.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets