No more 'business as usual' with audit and feedback interventions: towards an agenda for a reinvigorated intervention.
Ontology highlight
ABSTRACT: Background
Audit and feedback interventions in healthcare have been found to be effective, but there has been little progress with respect to understanding their mechanisms of action or identifying their key 'active ingredients.'Discussion
Given the increasing use of audit and feedback to improve quality of care, it is imperative to focus further research on understanding how and when it works best. In this paper, we argue that continuing the 'business as usual' approach to evaluating two-arm trials of audit and feedback interventions against usual care for common problems and settings is unlikely to contribute new generalizable findings. Future audit and feedback trials should incorporate evidence- and theory-based best practices, and address known gaps in the literature.Summary
We offer an agenda for high-priority research topics for implementation researchers that focuses on reviewing best practices for designing audit and feedback interventions to optimize effectiveness.
SUBMITTER: Ivers NM
PROVIDER: S-EPMC3896824 | biostudies-literature | 2014 Jan
REPOSITORIES: biostudies-literature
Publications
No more 'business as usual' with audit and feedback interventions: towards an agenda for a reinvigorated intervention.
Ivers Noah M NM Sales Anne A Colquhoun Heather H Michie Susan S Foy Robbie R Francis Jill J JJ Grimshaw Jeremy M JM
Implementation science : IS 20140117
<h4>Background</h4>Audit and feedback interventions in healthcare have been found to be effective, but there has been little progress with respect to understanding their mechanisms of action or identifying their key 'active ingredients.'<h4>Discussion</h4>Given the increasing use of audit and feedback to improve quality of care, it is imperative to focus further research on understanding how and when it works best. In this paper, we argue that continuing the 'business as usual' approach to evalu ...[more]
Similar Datasets
Project description:BackgroundAudit and feedback (A&F) interventions are one of the most common approaches for implementing evidence-based practices. A key barrier to more effective A&F interventions is the lack of a theory-guided approach to the accumulation of evidence. Recent interviews with theory experts identified 313 theory-informed hypotheses, spread across 30 themes, about how to create more effective A&F interventions. In the current survey, we sought to elicit from stakeholders which hypotheses were most likely to advance the field if studied further.MethodsFrom the list of 313, three members of the research team identified 216 that were clear and distinguishable enough for prioritization. A web-based survey was then sent to 211 A&F intervention stakeholders asking them to choose up to 50 'priority' hypotheses following the header "A&F interventions will be more effective if…". Analyses included frequencies of endorsement of the individual hypotheses and themes into which they were grouped.Results68 of the 211 invited participants responded to the survey. Seven hypotheses were chosen by > 50% of respondents, including A&F interventions will be more effective… "if feedback is provided by a trusted source"; "if recipients are involved in the design/development of the feedback intervention"; "if recommendations related to the feedback are based on good quality evidence"; "if the behaviour is under the control of the recipient"; "if it addresses barriers and facilitators (drivers) to behaviour change"; "if it suggests clear action plans"; and "if target/goal/optimal rates are clear and explicit". The most endorsed theme was Recipient Priorities (four hypotheses were chosen 92 times as a 'priority' hypotheses).ConclusionsThis work determined a set of hypotheses thought by respondents to be to be most likely to advance the field through future A&F intervention research. This work can inform a coordinated research agenda that may more efficiently lead to more effective A&F interventions.
| S-EPMC7805176 | biostudies-literature
Project description:BackgroundAudit and feedback (A&F) is a common strategy for helping health providers to implement evidence into practice. Despite being extensively studied, health care A&F interventions remain variably effective, with overall effect sizes that have not improved since 2003. Contributing to this stagnation is the fact that most health care A&F interventions have largely been designed without being informed by theoretical understanding from the behavioral and social sciences. To determine if the trend can be improved, the objective of this study was to develop a list of testable, theory-informed hypotheses about how to design more effective A&F interventions.MethodsUsing purposive sampling, semi-structured 60-90-min telephone interviews were conducted with experts in theories related to A&F from a range of fields (e.g., cognitive, health and organizational psychology, medical decision-making, economics). Guided by detailed descriptions of A&F interventions from the health care literature, interviewees described how they would approach the problem of designing improved A&F interventions. Specific, theory-informed hypotheses about the conditions for effective design and delivery of A&F interventions were elicited from the interviews. The resulting hypotheses were assigned by three coders working independently into themes, and categories of themes, in an iterative process.ResultsWe conducted 28 interviews and identified 313 theory-informed hypotheses, which were placed into 30 themes. The 30 themes included hypotheses related to the following five categories: A&F recipient (seven themes), content of the A&F (ten themes), process of delivery of the A&F (six themes), behavior that was the focus of the A&F (three themes), and other (four themes).ConclusionsWe have identified a set of testable, theory-informed hypotheses from a broad range of behavioral and social science that suggest conditions for more effective A&F interventions. This work demonstrates the breadth of perspectives about A&F from non-healthcare-specific disciplines in a way that yields testable hypotheses for healthcare A&F interventions. These results will serve as the foundation for further work seeking to set research priorities among the A&F research community.
| S-EPMC5622490 | biostudies-literature
Project description:Background. Audit and feedback (A/F) reports are one of the few knowledge translation activities that can effect change in physician behavior. In this study, we pilot-tested an endoscopist A/F report to elicit opinions about the proposed report's usability, acceptability and usefulness, and implications for knowledge translation. Methods. Semi-structured qualitative interviews were conducted with eleven endoscopists in Ontario, Canada. We tested an A/F report template comprising 9 validated, accepted colonoscopy quality indicators populated with simulated data. Interview transcripts were coded using techniques such as constant comparison and themes were identified inductively over several team meetings. Results. Four interrelated themes were identified: (1) overall perceptions of the A/F report; (2) accountability and consequences for poor performance; (3) motivation to change/improve skills; and (4) training for performance enhancement and available resources. The A/F report was well received; however, participants cited some possible threats to the report's effectiveness including the perceived threat of loss of privileges or licensing and the potential for the data to be dismissed. Conclusions. Participants agreed that A/F has the potential to improve colonoscopy performance. However, in order to be effective in changing physician behavior, A/F must be thoughtfully implemented with attention to the potential concerns of its recipients.
| S-EPMC5045985 | biostudies-literature
Project description:ObjectivesTo evaluate (1) the feasibility of an audit-feedback intervention to facilitate sports science journal policy change, (2) the reliability of the Transparency of Research Underpinning Social Intervention Tiers (TRUST) policy evaluation form, and (3) the extent to which policies of sports science journals support transparent and open research practices.MethodsWe conducted a cross-sectional, audit-feedback, feasibility study of transparency and openness standards of the top 38 sports science journals by impact factor. The TRUST form was used to evaluate journal policies support for transparent and open research practices. Feedback was provided to journal editors in the format of a tailored letter. Inter-rater reliability and agreement of the TRUST form was assessed using intraclass correlation coefficients and the standard error of measurement, respectively. Time-based criteria, fidelity of intervention delivery and qualitative feedback were used to determine feasibility.ResultsThe audit-feedback intervention was feasible based on the time taken to rate journals and provide tailored feedback. The mean (SD) score on the TRUST form (range 0-27) was 2.05 (1.99), reflecting low engagement with transparent and open practices. Inter-rater reliability of the overall score of the TRUST form was moderate [ICC (2,1) = 0.68 (95% CI 0.55-0.79)], with standard error of measurement of 1.17. However, some individual items had poor reliability.ConclusionPolicies of the top 38 sports science journals have potential for improved support for transparent and open research practices. The feasible audit-feedback intervention developed here warrants large-scale evaluation as a means to facilitate change in journal policies.RegistrationOSF ( https://osf.io/d2t4s/ ).
| S-EPMC9357245 | biostudies-literature
Project description:BackgroundAntimicrobial Stewardship (AMS), the set of actions to ensure appropriate antimicrobial use, is increasingly considered a multidisciplinary endeavour. However, it is unclear how Dutch hospital-based nurses envision their contribution to AMS.ObjectiveTo explore the views and visions of Dutch bedside nurses on their role regarding appropriate antimicrobial use.MethodsA qualitative study using semi-structured interviews was conducted. Fourteen bedside nurses in nine different Dutch hospitals participated. Data were analysed using a thematic content analysis.ResultsNurses considered their role regarding appropriate use of antibiotics as an integral part of their daily nursing practice. They envisioned their future role as an expansion of their current practice, improving or intensifying this contribution. Prompting review of antimicrobial treatment by nurses was seen as regular practice. Ward rounds were considered the best moment to exert their nursing role, by showing leadership in communicating about different aspects of AMS. Patient advocacy ("striving for the best possible care for their patient") appears to be a driver of the nursing contribution. Nurses perceived a shared responsibility with prescribers on certain aspects of the antimicrobial treatment and wished for a clarification of this role. Education and cognitive reminders such as antibiotic checklist to be used in ward rounds, can support the uptake of the nurses' role.ConclusionNurses envision their future role in AMS as an enhanced, elaborated and empowered version of their current daily practice. Education, formal acknowledgment and increased awareness of the nursing role, may advance the contributing role nurses already have.
| S-EPMC11361204 | biostudies-literature
Project description:The current state of much of the Wuhan pneumonia virus (severe acute respiratory syndrome coronavirus 2 [SARS-CoV-2]) research shows a regrettable lack of data sharing and considerable analytical obfuscation. This impedes global research cooperation, which is essential for tackling public health emergencies and requires unimpeded access to data, analysis tools, and computational infrastructure. Here, we show that community efforts in developing open analytical software tools over the past 10 years, combined with national investments into scientific computational infrastructure, can overcome these deficiencies and provide an accessible platform for tackling global health emergencies in an open and transparent manner. Specifically, we use all SARS-CoV-2 genomic data available in the public domain so far to (1) underscore the importance of access to raw data and (2) demonstrate that existing community efforts in curation and deployment of biomedical software can reliably support rapid, reproducible research during global health crises. All our analyses are fully documented at https://github.com/galaxyproject/SARS-CoV-2.
| S-EPMC7425854 | biostudies-literature
Project description:BackgroundThe use of clinical performance feedback to support quality improvement (QI) activities is based on the sound rationale that measurement is necessary to improve quality of care. However, concerns persist about the reliability of this strategy, known as Audit and Feedback (A&F) to support QI. If successfully implemented, A&F should reflect an iterative, self-regulating QI process. Whether and how real-world A&F initiatives result in this type of feedback loop are scarcely reported. This study aimed to identify barriers or facilitators to implementation in a team-based primary care context.MethodsSemi-structured interviews were conducted with key informants from team-based primary care practices in Ontario, Canada. At the time of data collection, practices could have received up to three iterations of the voluntary A&F initiative. Interviews explored whether, how, and why practices used the feedback to guide their QI activities. The Consolidated Framework for Implementation Research was used to code transcripts and the resulting frameworks were analyzed inductively to generate key themes.ResultsTwenty-five individuals representing 18 primary care teams participated in the study. Analysis of how the A&F intervention was used revealed that implementation reflected an incomplete feedback loop. Participation was facilitated by the reliance on an external resource to facilitate the practice audit. The frequency of feedback, concerns with data validity, the design of the feedback report, the resource requirements to participate, and the team relationship were all identified as barriers to implementation of A&F.ConclusionsThe implementation of a real-world, voluntary A&F initiative did not lead to desired QI activities despite substantial investments in performance measurement. In small primary care teams, it may take long periods of time to develop capacity for QI and future evaluations may reveal shifts in the implementation state of the initiative. Findings from the present study demonstrate that the potential mechanism of action of A&F may be deceptively clear; in practice, moving from measurement to action can be complex.
| S-EPMC6591867 | biostudies-literature
Project description:BackgroundAudit and feedback (A&F) is a widely used implementation strategy to evaluate and improve medical practice. The optimal design of an A&F system is uncertain and structured process evaluations are currently lacking. This study aimed to develop and validate a questionnaire to evaluate the use of automated A&F systems.MethodsBased on the Clinical Performance Feedback Intervention Theory (CP-FIT) and the REFLECT-52 (REassessing audit & Feedback interventions: a tooL for Evaluating Compliance with suggested besT practices) evaluation tool a questionnaire was designed for the purpose of evaluating automated A&F systems. A Rand-modified Delphi method was used to develop the process evaluation and obtain validation. Fourteen experts from different domains in primary care consented to participate and individually scored the questions on a 9-point Likert scale. Afterwards, the questions were discussed in a consensus meeting. After approval, the final questionnaire was compiled.ResultsA 34-question questionnaire composed of 57 items was developed and presented to the expert panel. The consensus meeting resulted in a selection of 31 questions, subdivided into 43 items. A final list of 30 questions consisting of 42 items was obtained.ConclusionA questionnaire consisting of 30 questions was drawn up for the assessment and improvement of automated A&F systems, based on CP-FIT and REFLECT-52 theory and approved by experts. Next steps will be piloting and implementation of the questionnaire.
| S-EPMC10998400 | biostudies-literature
Project description:BackgroundThis systematic review evaluates the effect of audit and feedback (A&F) interventions targeting antibiotic prescribing in primary care and examines factors that may explain the variation in effectiveness.MethodsRandomized controlled trials (RCTs) involving A&F interventions targeting antibiotic prescribing in primary care were included in the systematic review. Cochrane Central Register of Controlled Trials, MEDLINE, EMBASE, CINAHL, and ClinicalTrials.gov were searched up to May 2024. Trial, participant, and intervention characteristics were extracted independently by 2 researchers. Random effects meta-analyses of trials that compared interventions with and without A&F were conducted for 4 outcomes: (1) total antibiotic prescribing volume; (2) unnecessary antibiotic initiation; (3) excessive prescription duration, and (4) broad-spectrum antibiotic selection. A stratified analysis was also performed based on study characteristics and A&F intervention design features for total antibiotic volume.ResultsA total of 56 RCTs fit the eligibility criteria and were included in the meta-analysis. A&F was associated with an 11% relative reduction in antibiotic prescribing volume (N = 21 studies, rate ratio [RR] = 0.89; 95% confidence interval [CI]: .84, .95; I2 = 97); 23% relative reduction in unnecessary antibiotic initiation (N = 16 studies, RR = 0.77; 95% CI: .68, .87; I2 = 72); 13% relative reduction in prolonged duration of antibiotic course (N = 4 studies, RR = 0.87 95% CI: .81, .94; I2 = 86); and 17% relative reduction in broad-spectrum antibiotic selection (N = 17 studies, RR = 0.83 95% CI: .75, .93; I2 = 96).ConclusionsA&F interventions reduce antibiotic prescribing in primary care. However, heterogeneity was substantial, outcome definitions were not standardized across the trials, and intervention fidelity was not consistently assessed. Clinical Trials Registration. Prospero (CRD42022298297).
| S-EPMC11848270 | biostudies-literature
Project description:BackgroundAudit and feedback is a common intervention for supporting clinical behaviour change. Increasingly, health data are available in electronic format. Yet, little is known regarding if and how electronic audit and feedback (e-A&F) improves quality of care in practice.ObjectiveThe study aimed to assess the effectiveness of e-A&F interventions in a primary care and hospital context and to identify theoretical mechanisms of behaviour change underlying these interventions.MethodsIn August 2016, we searched five electronic databases, including MEDLINE and EMBASE via Ovid, and the Cochrane Central Register of Controlled Trials for published randomised controlled trials. We included studies that evaluated e-A&F interventions, defined as a summary of clinical performance delivered through an interactive computer interface to healthcare providers. Data on feedback characteristics, underlying theoretical domains, effect size and risk of bias were extracted by two independent review authors, who determined the domains within the Theoretical Domains Framework (TDF). We performed a meta-analysis of e-A&F effectiveness, and a narrative analysis of the nature and patterns of TDF domains and potential links with the intervention effect.ResultsWe included seven studies comprising of 81,700 patients being cared for by 329 healthcare professionals/primary care facilities. Given the extremely high heterogeneity of the e-A&F interventions and five studies having a medium or high risk of bias, the average effect was deemed unreliable. Only two studies explicitly used theory to guide intervention design. The most frequent theoretical domains targeted by the e-A&F interventions included 'knowledge', 'social influences', 'goals' and 'behaviour regulation', with each intervention targeting a combination of at least three. None of the interventions addressed the domains 'social/professional role and identity' or 'emotion'. Analyses identified the number of different domains coded in control arm to have the biggest role in heterogeneity in e-A&F effect size.ConclusionsGiven the high heterogeneity of identified studies, the effects of e-A&F were found to be highly variable. Additionally, e-A&F interventions tend to implicitly target only a fraction of known theoretical domains, even after omitting domains presumed not to be linked to e-A&F. Also, little evaluation of comparative effectiveness across trial arms was conducted. Future research should seek to further unpack the theoretical domains essential for effective e-A&F in order to better support strategic individual and team goals.
| S-EPMC5427645 | biostudies-literature