Dataset Information

Accounting for quality improvement during the conduct of embedded pragmatic clinical trials within healthcare systems: NIH Collaboratory case studies.

ABSTRACT: Embedded pragmatic clinical trials (ePCTs) and quality improvement (QI) activities often occur simultaneously within healthcare systems (HCSs). Embedded PCTs within HCSs are conducted to test interventions and provide evidence that may impact public health, health system operations, and quality of care. They are larger and more broadly generalizable than QI initiatives, and may generate what is considered high-quality evidence for potential use in care and clinical practice guidelines. QI initiatives often co-occur with ePCTs and address the same high-impact health questions, and this co-occurrence may dilute or confound the ability to detect change as a result of the ePCT intervention. During the design, pilot, and conduct phases of the large-scale NIH Collaboratory Demonstration ePCTs, many QI initiatives occurred at the same time within the HCSs. Although the challenges varied across the projects, some common, generalizable strategies and solutions emerged, and we share these as case studies. KEY LESSONS: Study teams often need to monitor, adapt, and respond to QI during design and the course of the trial. Routine collaboration between ePCT researchers and health systems stakeholders throughout the trial can help ensure research and QI are optimally aligned to support high-quality patient-centered care.

SUBMITTER: Tuzzio L

PROVIDER: S-EPMC8900087 | biostudies-literature | 2021 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Accounting for quality improvement during the conduct of embedded pragmatic clinical trials within healthcare systems: NIH Collaboratory case studies.

Tuzzio Leah L Meyers Catherine M CM Dember Laura M LM Grudzen Corita R CR Melnick Edward R ER Staman Karen L KL Huang Susan S SS Richards Julie J DeBar Lynn L Vazquez Miguel A MA Green Beverly B BB Coronado Gloria D GD Jarvik Jeffrey G JG Braciszewski Jordan J Ho P Michael PM Wells Barbara L BL James Kathryn K Toto Robert R D'Onofrio Gail G Volandes Angelo A Kuklinski Margaret R MR Catalano Richard F RF Sterling Stacy A SA Morse Erica F EF Curtis Lesley L Larson Eric B EB

Healthcare (Amsterdam, Netherlands) 20210623

Embedded pragmatic clinical trials (ePCTs) and quality improvement (QI) activities often occur simultaneously within healthcare systems (HCSs). Embedded PCTs within HCSs are conducted to test interventions and provide evidence that may impact public health, health system operations, and quality of care. They are larger and more broadly generalizable than QI initiatives, and may generate what is considered high-quality evidence for potential use in care and clinical practice guidelines. QI initia ...[more]

PMID: 34175091

Similar Datasets

Project description:BackgroundThe National Institutes of Health (NIH) Health Care Systems Research Collaboratory (NIH Collaboratory) seeks to produce generalizable knowledge about the conduct of pragmatic research in health systems. This analysis applied the PRECIS-2 pragmatic trial criteria to five NIH Collaboratory pragmatic trials to better understand 1) the pragmatic aspects of the design and implementation of treatments delivered in real world settings and 2) the usability of the PRECIS-2 criteria for assessing pragmatic features across studies and across time.Methods/designUsing the PRECIS-2 criteria, five pragmatic trials were each rated by eight raters. For each trial, we reviewed the original grant application and a required progress report written at the end of a 1-year planning period that included changes to the protocol or implementation approach. We calculated median scores and interrater reliability for each PRECIS domain and for the overall trial at both time points, as well as the differences in scores between the two time points. We also reviewed the rater comments associated with the scores.ResultsAll five trials were rated to be more pragmatic than explanatory, with comments indicating that raters generally perceived them to closely mirror routine clinical care across multiple domains. The PRECIS-2 domains for which the trials were, on average, rated as most pragmatic on the 1 to 5 scale at the conclusion of the planning period included primary analysis (mean = 4.7 (range = 4.5 to 4.9)), recruitment (4.3 (3.6 to 4.8)), eligibility (4.1 (3.4 to 4.8)), setting (4.1 (4.0 to 4.4)), follow-up (4.1 (3.4 to 4.9)), and primary outcome (4.1 (3.5 to 4.9)). On average, the less pragmatic domains were organization (3.3 (2.6 to 4.4)), flexibility of intervention delivery (3.5 (2.1-4.5)), and flexibility of intervention adherence (3.8 (2.8-4.5)). Interrater agreement was modest but statistically significant for four trials (Gwet's AC1 statistic range 0.23 to 0.40) and the intraclass correlation coefficient ranged from 0.05 to 0.31. Rating challenges included assigning a single score for domains that may relate to both patients and care settings (that is, eligibility or recruitment) and determining to what extent aspects of complex research interventions differ from usual care.ConclusionsThese five trials in diverse healthcare settings were rated as highly pragmatic using the PRECIS-2 criteria. Applying the tool generated insightful discussion about real-world design decisions but also highlighted challenges using the tool. PRECIS-2 raters would benefit from additional guidance about how to rate the interwoven patient and practice-level considerations that arise in pragmatic trials.Trial registrationsClinicaltrials.gov trial registrations: NCT02019225 , NCT01742065 , NCT02015455 , NCT02113592 , NCT02063867 .

Project description:IMPACT-AFib was an 80,000-patient randomized clinical trial implemented by five US insurance companies (health plans) aimed at increasing the use of oral anticoagulants by individuals with atrial fibrillation who were at high risk of stroke and not on treatment. The underlying thesis was that patients could be change agents to initiate prescribing discussions with their providers. We tested the effect of mailing information to both patients and their providers. We used administrative medical claims and pharmacy dispensing data to identify eligible patients, to randomize them to an early or delayed intervention, and to assess clinical outcomes. The core data were analysis-ready datasets each site had created and curated for the FDA's Sentinel System, supplemented by updated "fresh" pharmacy and enrollment data to ensure eligibility at the time of intervention. Following mutually agreed upon procedures, sites linked to additional internal source data to implement the intervention-educational information mailed to patients and their providers in the early intervention arm, and to providers of patients in the delayed intervention arm approximately 12 months later. The primary analysis compares the early intervention arm to the delayed intervention arm, prior to the delayed intervention being conducted (i.e. compares intervention to non-intervention). The endpoints of interest were evidence of initiation of anticoagulation (primary) as well as clinical endpoints, including stroke and hospitalization for bleeding. Major challenges, some unanticipated, identified during the planning phase include convening multi-stakeholder investigator teams and advisors, addressing ethical concerns about not intervening in a usual care comparison group, and identifying and avoiding interference with sites' routine programs that were similar to the intervention. Needs and challenges during the implementation phase included the fact that even limited site-specific programming greatly increased time and effort, the need to refresh research data extracts immediately before outreach to patients and providers, potential difficulty identifying low-cost medications such as warfarin that may not be reimbursed by health plans and so not discoverable in dispensing data, the need to develop workarounds when "providers" in claims data were facilities, difficulty addressing clustering of patients by provider because providers can have multiple identifiers within and between health plans, and the need to anticipate loss to follow up because of health plan disenrollment or change in benefits. As pragmatic trials begin to shape evidence generation within clinical practice, investigators should anticipate issues inherent to claims data and working with multiple large sites. In IMPACT-AFib, we found that investing in collaboration and communication among all parties throughout all phases of the study helped ensure common understanding, early identification of challenges, and streamlined actual implementation.

Project description:BackgroundRandomized controlled trial (RCT) trial designs exist on an explanatory-pragmatic spectrum, depending on the degree to which a study aims to address a question of efficacy or effectiveness. As conceptualized by Schwartz and Lellouch in 1967, an explanatory approach to trial design emphasizes hypothesis testing about the mechanisms of action of treatments under ideal conditions (efficacy), whereas a pragmatic approach emphasizes testing effectiveness of two or more available treatments in real-world conditions. Interest in, and the number of, pragmatic trials has grown substantially in recent years, with increased recognition by funders and stakeholders worldwide of the need for credible evidence to inform clinical decision-making. This increase has been accompanied by the onset of learning healthcare systems, as well as an increasing focus on patient-oriented research. However, pragmatic trials have ethical challenges that have not yet been identified or adequately characterized. The present study aims to explore the views of key stakeholders with respect to ethical issues raised by the design and conduct of pragmatic trials. It is embedded within a large, four-year project that seeks to develop guidance for the ethical design and conduct of pragmatic trials. As a first step, this study will address important gaps in the current empirical literature with respect to identifying a comprehensive range of ethical issues arising from the design and conduct of pragmatic trials. By opening up a broad range of topics for consideration within our parallel ethical analysis, we will extend the current debate, which has largely emphasized issues of consent, to the range of ethical considerations that may flow from specific design choices.MethodsSemi-structured interviews with key stakeholders (e.g. trialists, methodologists, lay members of study teams, bioethicists, and research ethics committee members), across multiple jurisdictions, identified based on their known experience and/or expertise with pragmatic trials.DiscussionWe expect that the study outputs will be of interest to a wide range of knowledge users including trialists, ethicists, research ethics committees, journal editors, regulators, healthcare policymakers, research funders and patient groups. All publications will adhere to the Tri-Agency Open Access Policy on Publications.

Dataset Information

Accounting for quality improvement during the conduct of embedded pragmatic clinical trials within healthcare systems: NIH Collaboratory case studies.

Publications

Accounting for quality improvement during the conduct of embedded pragmatic clinical trials within healthcare systems: NIH Collaboratory case studies.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets