Dataset Information

Development and usability testing of an audit and feedback tool for anesthesiologists.

ABSTRACT:

Background

We describe the creation and evaluation of a personal audit and feedback (A&F) tool for anesthesiologists.

Methods

A survey aimed at capturing barriers for personal improvement efforts and feedback preferences was administered to attending anesthesiologists. The results informed the design and implementation of 4 dashboards that display information on individual practice characteristics as well as comparative performance on several quality metrics. The dashboards' usability was then tested using the human-centered design framework.

Results

Anesthesiologists listed lack of information on current practice as the main barrier for improvement. Regarding usability, participants gave the dashboards an average score of 3.8 (scale 1-5) on consistency, learnability, and information organization, and performed the assigned tasks well, with an average score of 89% (range 79-100%).

Conclusions

We describe the design, implementation, and usability testing of an innovative tool that utilizes data derived from the electronic health record (EHR) system to provide A&F to anesthesiology providers.

SUBMITTER: Barbeito A

PROVIDER: S-EPMC6951966 | biostudies-literature | 2019 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Development and usability testing of an audit and feedback tool for anesthesiologists.

Barbeito Atilio A Segall Noa N

JAMIA open 20190204 1

<h4>Background</h4>We describe the creation and evaluation of a personal audit and feedback (A&F) tool for anesthesiologists.<h4>Methods</h4>A survey aimed at capturing barriers for personal improvement efforts and feedback preferences was administered to attending anesthesiologists. The results informed the design and implementation of 4 dashboards that display information on individual practice characteristics as well as comparative performance on several quality metrics. The dashboards' usabi ...[more]

PMID: 31984342

Similar Datasets

Project description:BackgroundAudit and Feedback (A&F) is a widely used quality improvement technique that measures clinicians' clinical performance and reports it back to them. Computerised A&F (e-A&F) system interfaces may consist of four key components: (1) Summaries of clinical performance; (2) Patient lists; (3) Patient-level data; (4) Recommended actions. There is a lack of evidence regarding how to best design e-A&F interfaces; establishing such evidence is key to maximising usability, and in turn improving patient safety.AimTo evaluate the usability of a novel theoretically-informed and research-led e-A&F system for primary care (the Performance Improvement plaN GeneratoR: PINGR).Objectives(1) Describe PINGR's design, rationale and theoretical basis; (2) Identify usability issues with PINGR; (3) Understand how these issues may interfere with the cognitive goals of end-users; (4) Translate the issues into recommendations for the user-centred design of e-A&F systems.MethodsEight experienced health system evaluators performed a usability inspection using an innovative hybrid approach consisting of five stages: (1) Development of representative user tasks, Goals, and Actions; (2) Combining Heuristic Evaluation and Cognitive Walkthrough methods into a single protocol to identify usability issues; (3) Consolidation of issues; (4) Severity rating of consolidated issues; (5) Analysis of issues according to usability heuristics, interface components, and Goal-Action structure.ResultsA final list of 47 issues were categorised into 8 heuristic themes. The most error-prone heuristics were 'Consistency and standards' (13 usability issues; 28% of the total) and 'Match between system and real world' (n=10, 21%). The recommended actions component of the PINGR interface had the most usability issues (n=21, 45%), followed by patient-level data (n=5, 11%), patient lists (n=4, 9%), and summaries of clinical performance (n=4, 9%). The most error-prone Actions across all user Goals were: (1) Patient selection from a list; (2) Data identification from a figure (both population-level and patient-level); (3) Disagreement with a system recommendation.ConclusionsBy contextualising our findings within the wider literature on health information system usability, we provide recommendations for the design of e-A&F system interfaces relating to their four key components, in addition to how they may be integrated within a system.

Project description:BackgroundAwareness of lung cancer screening remains low in the screening-eligible population, and when patients visit their clinician never having heard of lung cancer screening, engaging in shared decision making to arrive at an informed decision can be a challenge. Therefore, methods to effectively support both patients and clinicians to engage in these important discussions are essential. To facilitate shared decision making about lung cancer screening, effective methods to prepare patients to have these important discussions with their clinician are needed.ObjectiveOur objective is to develop a computer-tailored decision support tool that meets the certification criteria of the International Patient Decision Aid Standards instrument version 4.0 that will support shared decision making in lung cancer screening decisions.MethodsUsing a 3-phase process, we will develop and test a prototype of a computer-tailored decision support tool in a sample of lung cancer screening-eligible individuals. In phase I, we assembled a community advisory board comprising 10 screening-eligible individuals to develop the prototype. In phase II, we recruited a sample of 13 screening-eligible individuals to test the prototype for usability, acceptability, and satisfaction. In phase III, we are conducting a pilot randomized controlled trial (RCT) with 60 screening-eligible participants who have never been screened for lung cancer. Outcomes tested include lung cancer and screening knowledge, lung cancer screening health beliefs (perceived risk, perceived benefits, perceived barriers, and self-efficacy), perception of being prepared to engage in a patient-clinician discussion about lung cancer screening, occurrence of a patient-clinician discussion about lung cancer screening, and stage of adoption for lung cancer screening.ResultsPhases I and II are complete. Phase III is underway. As of July 15, 2017, 60 participants have been enrolled into the study, and have completed the baseline survey, intervention, and first follow-up survey. We expect to have results by December 31, 2017 and to have data analysis completed by March 1, 2018.ConclusionsResults from usability testing indicate that the computer-tailored decision support tool is easy to use, is helpful, and provides a satisfactory experience for the user. At the conclusion of phase III (pilot RCT), we will have preliminary effect sizes to inform a future fully powered RCT on changes in (1) knowledge about lung cancer and screening, (2) perceived risk of lung cancer, (3) perceived benefits of lung cancer screening, (4) perceived barriers to lung cancer screening, (5) self-efficacy for lung cancer screening, and (6) perceptions of being adequately prepared to engage in a discussion with their clinician about lung cancer screening.

Project description:ObjectivesWe describe the development of Your Multiple Sclerosis Questionnaire and present the real-world usability testing results of Your Multiple Sclerosis Questionnaire.MethodsThe Your Multiple Sclerosis Questionnaire tool was developed in four stages to collect feedback from people living with MS (plwMS), patient organizations, and clinicians on content, format, and applicability. To assess its usability, 13 clinicians across 7 countries completed an online survey after using the tool with plwMS in a total of 261 consultations from September, 2020 to July, 2021.ResultsThe initial Your Multiple Sclerosis Questionnaire version was based on findings from previous research developing MSProDiscuss™, a clinician-completed tool. Subsequently, insights from plwMS obtained during cognitive debriefing, patient councils and advisory boards led to changes including the addition of mood and sexual problems and the definition of relapse. All 13 clinicians completed the individual survey, whereas 10 clinicians completed the final survey. Clinicians "strongly agreed" or "agreed" that Your Multiple Sclerosis Questionnaire was easy to use and understand (98.5%; 257/261 patient consultations). The clinicians were willing to use the tool again with the same patient (98.1%; 256/261 patient consultations). All clinicians who completed the final survey (100%; 10/10) reported the tool to have a positive influence on their clinical practice, helped patients engage with their MS, facilitated discussion with patients, and complemented neurological assessment.ConclusionYour Multiple Sclerosis Questionnaire benefits both plwMS and clinicians by facilitating a structured discussion and engaging the plwMS to self-monitor and self-manage. Your Multiple Sclerosis Questionnaire is compatible with telemedicine practice and integration of the tool into electronic health records would enable tracking of the disease evolution and individual monitoring of MS symptoms over time.

Dataset Information

Development and usability testing of an audit and feedback tool for anesthesiologists.

Background

Methods

Results

Conclusions

Publications

Development and usability testing of an audit and feedback tool for anesthesiologists.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets