How well do critical care audit and feedback interventions adhere to best practice? Development and application of the REFLECT-52 evaluation tool.
Ontology highlight
ABSTRACT:
SUBMITTER: Foster M
PROVIDER: S-EPMC8369748 | biostudies-literature | 2021 Aug
REPOSITORIES: biostudies-literature
Publications
How well do critical care audit and feedback interventions adhere to best practice? Development and application of the REFLECT-52 evaluation tool.
Foster Madison M Presseau Justin J Podolsky Eyal E McIntyre Lauralyn L Papoulias Maria M Brehaut Jamie C JC
Implementation science : IS 20210817 1
<h4>Background</h4>Healthcare Audit and Feedback (A&F) interventions have been shown to be an effective means of changing healthcare professional behavior, but work is required to optimize them, as evidence suggests that A&F interventions are not improving over time. Recent published guidance has suggested an initial set of best practices that may help to increase intervention effectiveness, which focus on the "Nature of the desired action," "Nature of the data available for feedback," "Feedback ...[more]
Similar Datasets
Project description:BackgroundPrimary care is a critical partner for antimicrobial stewardship efforts given its high human antibiotic usage. Peer comparison audit and feedback (A&F) is often used to reduce inappropriate antibiotic prescribing. The design and implementation of A&F may impact its effectiveness. There are no best practice guidelines for peer comparison A&F in antibiotic prescribing in primary care.ObjectiveTo develop best practice guidelines for peer comparison A&F for antibiotic prescribing in primary care in high income countries by leveraging international expertise via the Joint Programming Initiative on Antimicrobial Resistance-Primary Care Antibiotic Audit and Feedback Network.MethodsWe used a modified Delphi process to achieve convergence of expert opinions on best practice statements for peer comparison A&F based on existing evidence and theory. Three rounds were performed, each with online surveys and virtual meetings to enable discussion and rating of each best practice statement. A five-point Likert scale was used to rate consensus with a median threshold score of 4 to indicate a consensus statement.ResultsThe final set of guidelines include 13 best practice statements in four categories: general considerations (n = 3), selecting feedback recipients (n = 1), data and indicator selection (n = 4), and feedback delivery (n = 5).ConclusionWe report an expert-derived best practice recommendations for designing and evaluating peer comparison A&F for antibiotic prescribing in primary care. These 13 statements can be used by A&F designers to optimize the impact of their quality improvement interventions, and improve antibiotic prescribing in primary care.
| S-EPMC10387210 | biostudies-literature
Project description:IntroductionPharmacists, as experts in medicines, are increasingly employed in general practices and undertake a range of responsibilities. Audit and feedback (A&F) interventions are effective in achieving behaviour change, including prescribing. The extent of pharmacist involvement in A&F interventions to influence prescribing is unknown. This review aimed to assess the effectiveness of A&F interventions involving pharmacists on prescribing in general practice compared with no A&F/usual care and to describe features of A&F interventions and pharmacist characteristics.MethodsElectronic databases (MEDLINE, EMBASE, CINAHL, Cochrane Central Register of Controlled Trials, (Social) Science Citation Indexes, ISI Web of Science) were searched (2012, 2019, 2020). Cochrane systematic review methods were applied to trial identification, selection, and risk of bias. Results were summarized descriptively and heterogeneity was assessed. A random-effects meta-analysis was conducted where studies were sufficiently homogenous in design and outcome.ResultsEleven cluster-randomized studies from 9 countries were included. Risk of bias across most domains was low. Interventions focussed on older patients, specific clinical area(s), or specific medications. Meta-analysis of 6 studies showed improved prescribing outcomes (pooled risk ratio: 0.78, 95% confidence interval: 0.64-0.94). Interventions including both verbal and written feedback or computerized decision support for prescribers were more effective. Pharmacists who received study-specific training, provided ongoing support to prescribers or reviewed prescribing for individual patients, contributed to more effective interventions.ConclusionsA&F interventions involving pharmacists can lead to small improvements in evidence-based prescribing in general practice settings. Future implementation of A&F within general practice should compare different ways of involving pharmacists to determine how to optimize effectiveness.PRISMA-compliant abstract included in Supplementary Material 1.
| S-EPMC10745261 | biostudies-literature
Project description:BackgroundAudit and feedback (A&F) is among the most widely used implementation strategies, providing healthcare professionals with summaries of their practice performance to prompt behaviour change and optimize care. Wide variability in effectiveness of A&F has spurred efforts to explore why some A&F interventions are more effective than others. Unpacking the variability of the content of A&F interventions in terms of their component behaviours change techniques (BCTs) may help advance our understanding of how A&F works best. This study aimed to systematically specify BCTs in A&F interventions targeting healthcare professional practice change.MethodsWe conducted a directed content analysis of intervention descriptions in 287 randomized trials included in an ongoing Cochrane systematic review update of A&F interventions (searched up to June 2020). Three trained researchers identified and categorized BCTs in all trial arms (treatment & control/comparator) using the 93-item BCT Taxonomy version 1. The original BCT definitions and examples in the taxonomy were adapted to include A&F-specific decision rules and examples. Two additional BCTs ('Education (unspecified)' and 'Feedback (unspecified)') were added, such that 95 BCTs were considered for coding.ResultsIn total, 47/95 BCTs (49%) were identified across 360 treatment arms at least once (median = 5.0, IQR = 2.3, range = 129 per arm). The most common BCTs were 'Feedback on behaviour' (present 89% of the time; e.g. feedback on drug prescribing), 'Instruction on how to perform the behaviour' (71%; e.g. issuing a clinical guideline), 'Social comparison' (52%; e.g. feedback on performance of peers), 'Credible source' (41%; e.g. endorsements from respected professional body), and 'Education (unspecified)' (31%; e.g. giving a lecture to staff). A total of 130/287 (45%) control/comparator arms contained at least one BCT (median = 2.0, IQR = 3.0, range = 0-15 per arm), of which the most common were identical to those identified in treatment arms.ConclusionsA&F interventions to improve healthcare professional practice include a moderate range of BCTs, focusing predominantly on providing behavioural feedback, sharing guidelines, peer comparison data, education, and leveraging credible sources. We encourage the use of our A&F-specific list of BCTs to improve knowledge of what is being delivered in A&F interventions. Our study provides a basis for exploring which BCTs are associated with intervention effectiveness.Trial registrationsN/A.
| S-EPMC10664600 | biostudies-literature
Project description:BackgroundIn Ontario, Canada, a government agency known as Ontario Health is responsible for making audit and feedback reports available to all family physicians to encourage ongoing quality improvement. The confidential report provides summary data on 3 key areas of practice: safe prescription, cancer screening, and diabetes management.ObjectiveThis report was redesigned to improve its usability in line with evidence. The objective of this study was to explore how the redesign was perceived, with an emphasis on recipients' understanding of the report and their engagement with it.MethodsWe conducted qualitative semistructured interviews with family physicians who had experience with both versions of the report recruited through purposeful and snowball sampling. We analyzed the transcripts following an emergent and iterative approach.ResultsSaturation was reached after 17 family physicians participated. In total, 2 key themes emerged as factors that affected the perceived usability of the report: alignment between the report and the recipients' expectations and capacity to engage in quality improvement. Family physicians expected the report and its quality indicators to reflect best practices and to be valid and accurate. They also expected the report to offer feedback on the clinical activities they perceived to be within their control to change. Furthermore, family physicians expected the goal of the report to be aligned with their perspective on feasible quality improvement activities. Most of these expectations were not met, limiting the perceived usability of the report. The capacity to engage with audit and feedback was hindered by several organizational and physician-level barriers, including the lack of fit with the existing workflow, competing priorities, time constraints, and insufficient skills for bridging the gaps between their data and the corresponding desired actions.ConclusionsDespite recognized improvements in the design of the report to better align with best practices, it was not perceived as highly usable. Improvements in the presentation of the data could not overcome misalignment with family physicians' expectations or the limited capacity to engage with the report. Integrating iterative evaluations informed by user-centered design can complement evidence-based guidance for implementation strategies. Creating a space for bringing together audit and feedback designers and recipients may help improve usability and effectiveness.
| S-EPMC9947922 | biostudies-literature
Project description:ObjectiveTo identify characteristics of antifungal prospective audit and feedback (PAF) and to compare rates of PAF recommendation and acceptance between antifungal and antibiotic agents.DesignRetrospective cohort study of antifungal and antibiotic audits by a children's hospital antimicrobial stewardship program (ASP) from November 1, 2020, to October 31, 2022.MethodsAntimicrobial audit data were retrieved from the ASP data warehouse. We characterized antifungal PAF using descriptive statistics. We then compared the overall rates of PAF recommendation and recommendation acceptance between antifungals and antibiotics. We also compared the differences in antifungal and antibiotic PAF recommendation and acceptance rates across various factors, including infectious problem, medical service, and recommendation type.ResultsOf 10,402 antimicrobial audits identified during the study period, 8,599 (83%) were for antibiotics and 1,803 (17%) were for antifungals. The highest antifungal recommendation rates were for liposomal amphotericin B, antifungals used for sepsis or respiratory tract infection, and antifungals prescribed in the cardiovascular intensive care unit. The rate of PAF recommendation was higher for antibiotics than for antifungals (29% vs 21%; P < .001); however, the rates of recommendation acceptance were similar. Recommendations to discontinue or for medication monitoring were more common for antifungals.ConclusionsOur analysis of antifungal PAF identified key opportunities to improve antifungal use, including the optimized use of specific agents and targeted use by certain medical services. Moreover, antifungal PAF, despite identifying fewer recommendations compared to antibiotic PAF, were associated with similarly high rates of acceptance, highlighting a promising opportunity for antifungal stewardship.
| S-EPMC10755142 | biostudies-literature
Project description:BackgroundAudit and feedback (A&F) is a common strategy for helping health providers to implement evidence into practice. Despite being extensively studied, health care A&F interventions remain variably effective, with overall effect sizes that have not improved since 2003. Contributing to this stagnation is the fact that most health care A&F interventions have largely been designed without being informed by theoretical understanding from the behavioral and social sciences. To determine if the trend can be improved, the objective of this study was to develop a list of testable, theory-informed hypotheses about how to design more effective A&F interventions.MethodsUsing purposive sampling, semi-structured 60-90-min telephone interviews were conducted with experts in theories related to A&F from a range of fields (e.g., cognitive, health and organizational psychology, medical decision-making, economics). Guided by detailed descriptions of A&F interventions from the health care literature, interviewees described how they would approach the problem of designing improved A&F interventions. Specific, theory-informed hypotheses about the conditions for effective design and delivery of A&F interventions were elicited from the interviews. The resulting hypotheses were assigned by three coders working independently into themes, and categories of themes, in an iterative process.ResultsWe conducted 28 interviews and identified 313 theory-informed hypotheses, which were placed into 30 themes. The 30 themes included hypotheses related to the following five categories: A&F recipient (seven themes), content of the A&F (ten themes), process of delivery of the A&F (six themes), behavior that was the focus of the A&F (three themes), and other (four themes).ConclusionsWe have identified a set of testable, theory-informed hypotheses from a broad range of behavioral and social science that suggest conditions for more effective A&F interventions. This work demonstrates the breadth of perspectives about A&F from non-healthcare-specific disciplines in a way that yields testable hypotheses for healthcare A&F interventions. These results will serve as the foundation for further work seeking to set research priorities among the A&F research community.
| S-EPMC5622490 | biostudies-literature
Project description:Prospective audit and feedback (PAF) is an effective strategy to optimize antimicrobial use in the critical care setting, yet whether skills gained during PAF influence future antimicrobial prescribing is uncertain. This multisite study demonstrates that knowledge learned during PAF is translated and incorporated into the practice of critical care physicians even when not supported by an antimicrobial stewardship program.
| S-EPMC4929488 | biostudies-literature
Project description:BackgroundAudit and feedback (A&F) interventions are one of the most common approaches for implementing evidence-based practices. A key barrier to more effective A&F interventions is the lack of a theory-guided approach to the accumulation of evidence. Recent interviews with theory experts identified 313 theory-informed hypotheses, spread across 30 themes, about how to create more effective A&F interventions. In the current survey, we sought to elicit from stakeholders which hypotheses were most likely to advance the field if studied further.MethodsFrom the list of 313, three members of the research team identified 216 that were clear and distinguishable enough for prioritization. A web-based survey was then sent to 211 A&F intervention stakeholders asking them to choose up to 50 'priority' hypotheses following the header "A&F interventions will be more effective if…". Analyses included frequencies of endorsement of the individual hypotheses and themes into which they were grouped.Results68 of the 211 invited participants responded to the survey. Seven hypotheses were chosen by > 50% of respondents, including A&F interventions will be more effective… "if feedback is provided by a trusted source"; "if recipients are involved in the design/development of the feedback intervention"; "if recommendations related to the feedback are based on good quality evidence"; "if the behaviour is under the control of the recipient"; "if it addresses barriers and facilitators (drivers) to behaviour change"; "if it suggests clear action plans"; and "if target/goal/optimal rates are clear and explicit". The most endorsed theme was Recipient Priorities (four hypotheses were chosen 92 times as a 'priority' hypotheses).ConclusionsThis work determined a set of hypotheses thought by respondents to be to be most likely to advance the field through future A&F intervention research. This work can inform a coordinated research agenda that may more efficiently lead to more effective A&F interventions.
| S-EPMC7805176 | biostudies-literature
Project description:BackgroundLaboratory tests and transfusions are sometimes ordered inappropriately, particularly in the critical care setting, which sees frequent use of both. Audit and Feedback (A&F) is a potentially useful intervention for modifying healthcare provider behaviors, but its application to the complex, team-based environment of critical care is not well understood. We conducted a systematic review of the literature on A&F interventions for improving test or transfusion ordering in the critical care setting.MethodsFive databases, two registries, and the bibliographies of relevant articles were searched. We included critical care studies that assessed the use of A&F targeting healthcare provider behaviors, alone or in combination with other interventions to improve test and transfusion ordering, as compared to historical practice, no intervention, or another healthcare behaviour change intervention. Studies were included only if they reported laboratory test or transfusion orders, or the appropriateness of orders, as outcomes. There were no restrictions based on study design, date of publication, or follow-up time. Intervention characteristics and absolute differences in outcomes were summarized. The quality of individual studies was assessed using a modified version of the Effective Practice and Organisation of Care Cochrane Review Group's criteria.ResultsWe identified 16 studies, including 13 uncontrolled before-after studies, one randomized controlled trial, one controlled before-after study, and one controlled clinical trial (quasi-experimental). These studies described 17 interventions, mostly (88%) multifaceted interventions with an A&F component. Feedback was most often provided in a written format only (41%), more than once (53%), and most often only provided data aggregated to the group-level (41%). Most studies saw a change in the hypothesized direction, but not all studies provided statistical analyses to formally test improvement. Overall study quality was low, with studies often lacking a concurrent control group.ConclusionsOur review summarizes characteristics of A&F interventions implemented in the critical care context, points to some mechanisms by which A&F might be made more effective in this setting, and provides an overview of how the appropriateness of orders was reported. Our findings suggest that A&F can be effective in the context of critical care; however, further research is required to characterize approaches that optimize the effectiveness in this setting alongside more rigorous evaluation methods.Trial registrationPROSPERO CRD42016051941.
| S-EPMC7303577 | biostudies-literature
Project description:BackgroundThis systematic review evaluates the effect of audit and feedback (A&F) interventions targeting antibiotic prescribing in primary care and examines factors that may explain the variation in effectiveness.MethodsRandomized controlled trials (RCTs) involving A&F interventions targeting antibiotic prescribing in primary care were included in the systematic review. Cochrane Central Register of Controlled Trials, MEDLINE, EMBASE, CINAHL, and ClinicalTrials.gov were searched up to May 2024. Trial, participant, and intervention characteristics were extracted independently by 2 researchers. Random effects meta-analyses of trials that compared interventions with and without A&F were conducted for 4 outcomes: (1) total antibiotic prescribing volume; (2) unnecessary antibiotic initiation; (3) excessive prescription duration, and (4) broad-spectrum antibiotic selection. A stratified analysis was also performed based on study characteristics and A&F intervention design features for total antibiotic volume.ResultsA total of 56 RCTs fit the eligibility criteria and were included in the meta-analysis. A&F was associated with an 11% relative reduction in antibiotic prescribing volume (N = 21 studies, rate ratio [RR] = 0.89; 95% confidence interval [CI]: .84, .95; I2 = 97); 23% relative reduction in unnecessary antibiotic initiation (N = 16 studies, RR = 0.77; 95% CI: .68, .87; I2 = 72); 13% relative reduction in prolonged duration of antibiotic course (N = 4 studies, RR = 0.87 95% CI: .81, .94; I2 = 86); and 17% relative reduction in broad-spectrum antibiotic selection (N = 17 studies, RR = 0.83 95% CI: .75, .93; I2 = 96).ConclusionsA&F interventions reduce antibiotic prescribing in primary care. However, heterogeneity was substantial, outcome definitions were not standardized across the trials, and intervention fidelity was not consistently assessed. Clinical Trials Registration. Prospero (CRD42022298297).
| S-EPMC11848270 | biostudies-literature