Corrigendum to: Interrupted time series regression for the evaluation of public health interventions: a tutorial.
Ontology highlight
ABSTRACT:
SUBMITTER: Bernal JL
PROVIDER: S-EPMC7750921 | biostudies-literature | 2020 Aug
REPOSITORIES: biostudies-literature
Publications
Corrigendum to: Interrupted time series regression for the evaluation of public health interventions: a tutorial.
Bernal James Lopez JL Cummins Steven S Gasparrini Antonio A
International journal of epidemiology 20200801 4
Similar Datasets
Project description:Interrupted time series (ITS) analysis is a valuable study design for evaluating the effectiveness of population-level health interventions that have been implemented at a clearly defined point in time. It is increasingly being used to evaluate the effectiveness of interventions ranging from clinical therapy to national public health legislation. Whereas the design shares many properties of regression-based approaches in other epidemiological studies, there are a range of unique features of time series data that require additional methodological considerations. In this tutorial we use a worked example to demonstrate a robust approach to ITS analysis using segmented regression. We begin by describing the design and considering when ITS is an appropriate design choice. We then discuss the essential, yet often omitted, step of proposing the impact model a priori. Subsequently, we demonstrate the approach to statistical analysis including the main segmented regression model. Finally we describe the main methodological issues associated with ITS analysis: over-dispersion of time series data, autocorrelation, adjusting for seasonal trends and controlling for time-varying confounders, and we also outline some of the more complex design adaptations that can be used to strengthen the basic ITS design.
| S-EPMC5407170 | biostudies-literature
Project description:INTRODUCTION:An interrupted time series (ITS) design is an important observational design used to examine the effects of an intervention or exposure. This design has particular utility in public health where it may be impracticable or infeasible to use a randomised trial to evaluate health system-wide policies, or examine the impact of exposures (such as earthquakes). There have been relatively few studies examining the design characteristics and statistical methods used to analyse ITS designs. Further, there is a lack of guidance to inform the design and analysis of ITS studies.This is the first study in a larger project that aims to provide tools and guidance for researchers in the design and analysis of ITS studies. The objectives of this study are to (1) examine and report the design characteristics and statistical methods used in a random sample of contemporary ITS studies examining public health interventions or exposures that impact on health-related outcomes, and (2) create a repository of time series data extracted from ITS studies. Results from this study will inform the remainder of the project which will investigate the performance of a range of commonly used statistical methods, and create a repository of input parameters required for sample size calculation. METHODS AND ANALYSIS:We will collate 200 ITS studies evaluating public health interventions or the impact of exposures. ITS studies will be identified from a search of the bibliometric database PubMed between the years 2013 and 2017, combined with stratified random sampling. From eligible studies, we will extract study characteristics, details of the statistical models and estimation methods, effect metrics and parameter estimates. Further, we will extract the time series data when available. We will use systematic review methods in the screening, application of inclusion and exclusion criteria, and extraction of data. Descriptive statistics will be used to summarise the data. ETHICS AND DISSEMINATION:Ethics approval is not required since information will only be extracted from published studies. Dissemination of the results will be through peer-reviewed publications and presentations at conferences. A repository of data extracted from the published ITS studies will be made publicly available.
| S-EPMC6352832 | biostudies-literature
Project description:BACKGROUND:When randomisation is not possible, interrupted time series (ITS) design has increasingly been advocated as a more robust design to evaluating health system quality improvement (QI) interventions given its ability to control for common biases in healthcare QI. However, there is a potential risk of producing misleading results when this rather robust design is not used appropriately. We performed a methodological systematic review of the literature to investigate the extent to which the use of ITS has followed best practice standards and recommendations in the evaluation of QI interventions. METHODS:We searched multiple databases from inception to June 2018 to identify QI intervention studies that were evaluated using ITS. There was no restriction on date, language and participants. Data were synthesised narratively using appropriate descriptive statistics. The risk of bias for ITS studies was assessed using the Cochrane Effective Practice and Organisation of Care standard criteria. The systematic review protocol was registered in PROSPERO (registration number: CRD42018094427). RESULTS:Of 4061 potential studies and 2028 unique records screened for inclusion, 120 eligible studies assessed eight QI strategies and were from 25 countries. Most studies were published since 2010 (86.7%), reported data using monthly interval (71.4%), used ITS without a control (81%) and modelled data using segmented regression (62.5%). Autocorrelation was considered in 55% of studies, seasonality in 20.8% and non-stationarity in 8.3%. Only 49.2% of studies specified the ITS impact model. The risk of bias was high or very high in 72.5% of included studies and did not change significantly over time. CONCLUSIONS:The use of ITS in the evaluation of health system QI interventions has increased considerably over the past decade. However, variations in methodological considerations and reporting of ITS in QI remain a concern, warranting a need to develop and reinforce formal reporting guidelines to improve its application in the evaluation of health system QI interventions.
| S-EPMC7559052 | biostudies-literature
Project description:BackgroundA classic methodology used in evaluating the impact of health policy interventions is interrupted time-series (ITS) analysis, applying a quasi-experimental design that uses both pre- and post-policy data without randomization. In this paper, we took a simulation-based approach to estimating intervention effects under different assumptions.MethodsEach of the simulated mortality rates contained a linear time trend, seasonality, autoregressive, and moving-average terms. The simulations of the policy effects involved three scenarios: 1) immediate-level change only, 2) immediate-level and slope change, and 3) lagged-level and slope change. The estimated effects and biases of these effects were examined via three matched generalized additive mixed models, each of which used two different approaches: 1) effects based on estimated coefficients (estimated approach), and 2) effects based on predictions from models (predicted approach). The robustness of these two approaches was further investigated assuming misspecification of the models.ResultsWhen one simulated dataset was analyzed with the matched model, the two analytical approaches produced similar estimates. However, when the models were misspecified, the number of deaths prevented, estimated using the predicted vs. estimated approaches, were very different, with the predicted approach yielding estimates closer to the real effect. The discrepancy was larger when the policy was applied early in the time-series.ConclusionEven when the sample size appears to be large enough, one should still be cautious when conducting ITS analyses, since the power also depends on when in the series the intervention occurs. In addition, the intervention lagged effect needs to be fully considered at the study design stage (i.e., when developing the models).
| S-EPMC9429656 | biostudies-literature
Project description:ObjectiveTo discuss the study design and data analysis for three-phase interrupted time series (ITS) studies to evaluate the impact of health policy, systems, or environmental interventions. Simulation methods are used to conduct power and sample size calculation for these studies.MethodsWe consider the design and analysis of three-phase ITS studies using a study funded by National Institutes of Health as an exemplar. The design and analysis of both one-arm and two-arm three-phase ITS studies are introduced.ResultsA simulation-based approach, with ready-to-use computer programs, was developed to determine the power for two types of three-phase ITS studies. Simulations were conducted to estimate the power of segmented autoregressive (AR) error models when autocorrelation ranged from -0.9 to 0.9 with various effect sizes. The power increased as the sample size or the effect size increased. The power to detect the same effect sizes varied largely, depending on testing level change, trend changes, or both.ConclusionThis article provides a convenient tool for investigators to generate sample sizes to ensure sufficient statistical power when three-phase ITS study design is implemented.
| S-EPMC7028460 | biostudies-literature
Project description:BackgroundReducing maternal mortality remains a global priority. In 2000, the United Nations Member States pledged to work towards a series of Millennium Development Goals (MDGs), in which the fifth target was to reduce maternal mortality ratio by 75% from 1990 to 2015. The Chinese government introduced Basic Public Health Service project in 2009 to the further improvement of maternal health services and reduction in maternal mortality. China had achieved the goal of MDG5 1 year ahead of the schedule in 2014, but the effects of the project on reducing maternal mortality were rarely evaluated with robust methods.MethodsWe conducted a longitudinal study on maternal mortality ratio by extracting mortality data from the National Maternal Mortality Surveillance System (1991-2016) and maternal health services measures from the China health statistic yearbook (2001-2016). We utilized the segmented linear regression model to assess changes and trends of maternal mortality ratio and maternal health services before and after the introduction of Basic Public Health Service project. Pearson correlation analysis was conducted to measure the strength of association between the maternal mortality ratio and maternal health services.ResultsThe yearly trend change of national maternal mortality ratio was - 1.76 (p < 0.01) after the introduction of Basic Public Health Service project in 2009, while the yearly trend change of maternal health record establish rate, prenatal examination rate, postpartum visit rate was 0.77 (p < 0.01), 0.61 (p < 0.01) and 0.83 (p < 0.01) separately. The negative correlations were also found between national maternal mortality ratio and prenatal examination rate (r = - 0.95, p < 0.01), maternal health record establish rate (r = - 0.93, p < 0.01) and postpartum visit rate (r = - 0.92, p < 0.01).ConclusionsThe Basic Public Health Service project was found to be associated with the improvements in the maternal health services and reduction in maternal mortality. The design and implementation of the project may serve as a positive example for other developing countries. Continued monitoring and assessment of project effects should be stressed.
| S-EPMC6595598 | biostudies-literature
Project description:Background and purposeRecent methodological reviews of evaluations of behaviour change interventions in public health have highlighted that the decay in effectiveness over time has been mostly overlooked, potentially leading to suboptimal decision-making. While, in principle, discrete-time Markov chains-the most commonly used modelling approach-can be adapted to account for decay in effectiveness, this framework inherently lends itself to strong model simplifications. The application of formal and more appropriate modelling approaches has been supported, but limited progress has been made to date. The purpose of this paper is to encourage this shift by offering a practical guide on how to model decay in effectiveness using a continuous-time Markov chain (CTMC)-based approach.MethodsA CTMC approach is demonstrated, with a contextualized tutorial being presented to facilitate learning and uptake. A worked example based on the stylized case study in physical activity promotion is illustrated with accompanying R code.DiscussionThe proposed framework presents a relatively small incremental change from the current modelling practice. CTMC represents a technical solution which, in absence of relevant data, allows for formally testing the sensitivity of results to assumptions regarding the long-term sustainability of intervention effects and improving model transparency.ConclusionsThe use of CTMC should be considered in evaluations where decay in effectiveness is likely to be a key factor to consider. This would enable more robust model-based evaluations of population-level programmes to promote behaviour change and reduce the uncertainty surrounding the decision to invest in these public health interventions.
| S-EPMC9395462 | biostudies-literature
Project description:BackgroundThe long-term impact of COVID-19-associated public health interventions on zoonotic and vector-borne infectious diseases (ZVBs) remains uncertain. This study sought to examine the changes in ZVBs in China during the COVID-19 pandemic and predict their future trends.MethodsMonthly incidents of seven ZVBs (Hemorrhagic fever with renal syndrome [HFRS], Rabies, Dengue fever [DF], Human brucellosis [HB], Leptospirosis, Malaria, and Schistosomiasis) were gathered from January 2004 to July 2023. An autoregressive fractionally integrated moving average (ARFIMA) by incorporating the COVID-19-associated public health intervention variables was developed to evaluate the long-term effectiveness of interventions and forecast ZVBs epidemics from August 2023 to December 2025.ResultsOver the study period, there were 1,599,647 ZVBs incidents. HFRS and rabies exhibited declining trends, HB showed an upward trajectory, while the others remained relatively stable. The ARFIMA, incorporating a pulse pattern, estimated the average monthly number of changes of - 83 (95% confidence interval [CI] - 353-189) cases, - 3 (95% CI - 33-29) cases, - 468 (95% CI - 1531-597) cases, 2191 (95% CI 1056-3326) cases, 7 (95% CI - 24-38) cases, - 84 (95% CI - 222-55) cases, and - 214 (95% CI - 1036-608) cases for HFRS, rabies, DF, HB, leptospirosis, malaria, and schistosomiasis, respectively, although these changes were not statistically significant besides HB. ARFIMA predicted a decrease in HB cases between August 2023 and December 2025, while indicating a relative plateau for the others.ConclusionsChina's dynamic zero COVID-19 strategy may have exerted a lasting influence on HFRS, rabies, DF, malaria, and schistosomiasis, beyond immediate consequences, but not affect HB and leptospirosis. ARFIMA emerges as a potent tool for intervention analysis, providing valuable insights into the sustained effectiveness of interventions. Consequently, the application of ARFIMA contributes to informed decision-making, the design of effective interventions, and advancements across various fields.
| S-EPMC10799468 | biostudies-literature
Project description:BackgroundDespite major primary health care (PHC) reforms in China with the 2009 launch of the National Essential Public Health Service Package, the country experiences many challenges in improving the management of non-communicable diseases in PHC facilities. "EMERALD" is a multifaceted implementation strategy to strengthen the management of hypertension and type-2 diabetes mellitus (T2DM) in PHC facilities. The study aims to: (1) examine the effectiveness of EMERALD in improving hypertension and T2DM management; (2) evaluate the implementation of the interventions; and (3) use the study findings to model the long-term health economic impact of the interventions.MethodsThe EMERALD intervention components include: (1) empowerment for PHC providers through training and capacity building; (2) empowerment for patient communities through multi-media health education; and (3) empowerment for local health administrators through health data monitoring and strengthening governance of local PHC programs. An interrupted time series design will be used to determine the effectiveness of the interventions based on routinely collected health data extracted from local health information systems. The primary effectiveness outcome is the guideline-recommended treatment rates for people with hypertension and T2DM. Secondary effectiveness outcomes include hypertension and T2DM diagnosis and control rates, and enrolment and adherence rates to the recommended care processes in the National Essential Public Health Service Package. A mixed-methods process evaluation will be conducted to evaluate the implementation of the interventions, including the reach of the target population, adequacy of adoption, level of implementation fidelity, and maintenance. Qualitative interviews with policy makers, health administrators, PHC providers, and patients with hypertension and/or T2DM will be conducted to further identify factors influencing the implementation. In addition, health economic modelling will be performed to explore the long-term incremental costs and benefits of the interventions.DiscussionThis study is expected to generate important evidence on the effectiveness, implementation, and health economic impact of complex PHC interventions to strengthen the primary care sector's contribution to addressing the growing burden of non-communicable diseases in China.Trial registrationThe study has been registered on Chinese Clinical Trial Registry at https://www.chictr.org.cn/ (Registration number ChiCTR2400082036, on March 19th 2024).
| S-EPMC11414291 | biostudies-literature
Project description:BACKGROUND:Rapid evaluation was at the heart of National Health Service England's evaluation strategy of the new models of care vanguard programme. This was to facilitate the scale and spread of successful models of care throughout the health & social care system. The aim of this paper is to compare the findings of the two evaluations of the Enhanced health in Care Homes (EHCH) vanguard in Gateshead, one using a smaller data set for rapidity and one using a larger longitudinal data set and to investigate the implications of the use of rapid evaluations using interrupted time series (ITS) methods. METHODS:A quasi-experimental design study in the form of an ITS was used to evaluate the impact of the vanguard on secondary care use. Two different models are presented differing by timeframes only. The short-term model consisted of data for 11 months data pre and 20 months post vanguard. The long-term model consisted of data for 23 months pre and 34 months post vanguard. RESULTS:The cost consequences, including the cost of running the EHCH vanguard, were estimated using both a single tariff non-elective admissions methodology and a tariff per bed day methodology. The short-term model estimated a monthly cost increase of £73,408 using a single tariff methodology. When using a tariff per bed day, there was an estimated monthly cost increase of £14,315. The long-term model had, using a single tariff for non-elective admissions, an overall cost increase of £7576 per month. However, when using a tariff per bed-days, there was an estimated monthly cost reduction of £57,168. CONCLUSIONS:Although it is acknowledged that there is often a need for rapid evaluations in order to identify "quick wins" and to expedite learning within health and social care systems, we conclude that this may not be appropriate for quasi-experimental designs estimating effect using ITS for complex interventions. Our analyses suggests that care must be taken when conducting and interpreting the results of short-term evaluations using ITS methods, as they may produce misleading results and may lead to a misallocation of resources.
| S-EPMC6911271 | biostudies-literature