Dataset Information

Usability of a perioperative medication-related clinical decision support software application: a randomized controlled trial.

ABSTRACT:

Objective

We developed a comprehensive, medication-related clinical decision support (CDS) software prototype for use in the operating room. The purpose of this study was to compare the usability of the CDS software to the current standard electronic health record (EHR) medication administration and documentation workflow.

Materials and methods

The primary outcome was the time taken to complete all simulation tasks. Secondary outcomes were the total number of mouse clicks and the total distance traveled on the screen in pixels. Forty participants were randomized and assigned to complete 7 simulation tasks in 1 of 2 groups: (1) the CDS group (n = 20), who completed tasks using the CDS and (2) the Control group (n = 20), who completed tasks using the standard medication workflow with retrospective manual documentation in our anesthesia information management system. Blinding was not possible. We video- and audio-recorded the participants to capture quantitative data (time on task, mouse clicks, and pixels traveled on the screen) and qualitative data (think-aloud verbalization).

Results

The CDS group mean total task time (402.2 ± 85.9 s) was less than the Control group (509.8 ± 103.6 s), with a mean difference of 107.6 s (95% confidence interval [CI], 60.5-179.5 s, P < .001). The CDS group used fewer mouse clicks (26.4 ± 4.5 clicks) than the Control group (56.0 ± 15.0 clicks) with a mean difference of 29.6 clicks (95% CI, 23.2-37.6, P < .001). The CDS group had fewer pixels traveled on the computer monitor (59.5 ± 20.0 thousand pixels) than the Control group (109.3 ± 40.8 thousand pixels) with a mean difference of 49.8 thousand pixels (95% CI, 33.0-73.7, P < .001).

Conclusions

The perioperative medication-related CDS software prototype substantially outperformed standard EHR workflow by decreasing task time and improving efficiency and quality of care in a simulation setting.

SUBMITTER: Nanji KC

PROVIDER: S-EPMC9277651 | biostudies-literature | 2022 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Usability of a perioperative medication-related clinical decision support software application: a randomized controlled trial.

Nanji Karen C KC Garabedian Pamela M PM Langlieb Marin E ME Rui Angela A Tabayoyong Leo L LL Sampson Michael M Deng Hao H Boxwala Aziz A Minehart Rebecca D RD Bates David W DW

Journal of the American Medical Informatics Association : JAMIA 20220701 8

<h4>Objective</h4>We developed a comprehensive, medication-related clinical decision support (CDS) software prototype for use in the operating room. The purpose of this study was to compare the usability of the CDS software to the current standard electronic health record (EHR) medication administration and documentation workflow.<h4>Materials and methods</h4>The primary outcome was the time taken to complete all simulation tasks. Secondary outcomes were the total number of mouse clicks and the ...[more]

PMID: 35575780

Similar Datasets

Project description:BackgroundOsteoporosis medication use is suboptimal. Simple interventions personalized to a patients' stage of readiness are needed to encourage osteoporosis medication use.ObjectivesTo estimate interrelationships of sociodemographic factors, perceived fracture risk, health literacy, receipt of medication information, medication trust and readiness to use osteoporosis medication; and apply observed relationships to inform design specifications for a clinical decision support application that can be used for personalized patient counseling.MethodsData from a national sample of older women (n = 1759) with self-reported history of fractures and no current use of osteoporosis medication treatment were used to estimate an acceptable path model that describes associations among key sociodemographic characteristics, health literacy, perceived fracture risk, receipt of osteoporosis medication information within the past year, trust in osteoporosis medications, and readiness to use osteoporosis medication. Path model results were used to inform an application for personalized patient counseling that can be easily integrated into clinical decision support systems.ResultsIncreased age (β = 0.13), trust for medications (β = 0.12), higher perceived fracture risk (β = 0.21), and having received medication information within the past year (β = 0.21) were all positively associated with readiness to use osteoporosis medication (p < 0.0001). Whereas, health literacy (β = -0.09) was inversely associated with readiness to use osteoporosis medication (p < 0.0001). Using these results, a brief 6-item question set was constructed for simple integration into clinical decision support applications. Patient responses were used to inform a provider dashboard that integrates a patient's stage of readiness for osteoporosis medication use, predictors of readiness, and personalized counseling points appropriate to their stage of readiness.ConclusionContent of counseling strategies must be aligned with a patient's stage of readiness to use treatment. Path modeling can be effectively used to identify factors for inclusion in an evidenced-based clinical decision support application designed to assist providers with personalized patient counseling and osteoporosis medication use decisions.

Project description:BackgroundAn increasing number of diagnostic decision support systems (DDSS) exist to support patients and physicians in establishing the correct diagnosis as early as possible. However, little evidence exists that supports the effectiveness of these DDSS. The objectives were to compare the diagnostic accuracy of medical students, with and without the use of a DDSS, and the diagnostic accuracy of the DDSS system itself, regarding the typical rheumatic diseases and to analyze the user experience.MethodsA total of 102 medical students were openly recruited from a university hospital and randomized (unblinded) to a control group (CG) and an intervention group (IG) that used a DDSS (Ada - Your Health Guide) to create an ordered diagnostic hypotheses list for three rheumatic case vignettes. Diagnostic accuracy, measured as the presence of the correct diagnosis first or at all on the hypothesis list, was the main outcome measure and evaluated for CG, IG, and DDSS.ResultsThe correct diagnosis was ranked first (or was present at all) in CG, IG, and DDSS in 37% (40%), 47% (55%), and 29% (43%) for the first case; 87% (94%), 84% (100%), and 51% (98%) in the second case; and 35% (59%), 20% (51%), and 4% (51%) in the third case, respectively. No significant benefit of using the DDDS could be observed. In a substantial number of situations, the mean probabilities reported by the DDSS for incorrect diagnoses were actually higher than for correct diagnoses, and students accepted false DDSS diagnostic suggestions. DDSS symptom entry greatly varied and was often incomplete or false. No significant correlation between the number of symptoms extracted and diagnostic accuracy was seen. It took on average 7 min longer to solve a case using the DDSS. In IG, 61% of students compared to 90% in CG stated that they could imagine using the DDSS in their future clinical work life.ConclusionsThe diagnostic accuracy of medical students was superior to the DDSS, and its usage did not significantly improve students' diagnostic accuracy. DDSS usage was time-consuming and may be misleading due to prompting wrong diagnoses and probabilities.Trial registrationDRKS.de, DRKS00024433 . Retrospectively registered on February 5, 2021.

Project description:BackgroundThe 3-Minute Diagnostic Interview for Confusion Assessment Method-Defined Delirium (3D-CAM) is an instrument specially developed for the assessment of delirium in general wards, with high reported sensitivity and specificity. However, the use of the 3D-CAM by bedside nurses in routine practice showed relatively poor usability, with multiple human errors during assessment.ObjectiveThis study aimed to develop a mobile app-based delirium assessment tool based on the 3D-CAM and evaluate its usability among older patients by bedside nurses.MethodsThe Delirium Assessment Tool With Decision Support Based on the 3D-CAM (3D-DST) was developed to address existing issues of the 3D-CAM and optimize the assessment process. Following a randomized crossover design, questionnaires were used to evaluate the usability of the 3D-DST among older adults by bedside nurses. Meanwhile, the performances of both the 3D-DST and the 3D-CAM paper version, including the assessment completion rate, time required for completing the assessment, and the number of human errors made by nurses during assessment, were recorded, and their differences were compared.ResultsThe 3D-DST included 3 assessment modules, 9 evaluation interfaces, and 16 results interfaces, with built-in reminders to guide nurses in completing the delirium assessment. In the usability testing, a total of 432 delirium assessments (216 pairs) on 148 older adults were performed by 72 bedside nurses with the 3D-CAM paper version and the 3D-DST. Compared to the 3D-CAM paper version, the mean usability score was significantly higher when using the 3D-DST (4.35 vs 3.40; P<.001). The median scores of the 6 domains of the satisfactory evaluation questionnaire for nurses using the 3D-CAM paper version and the 3D-DST were above 2.83 and 4.33 points, respectively (P<.001). The average time for completing the assessment reduced by 2.1 minutes (4.4 vs 2.3 min; P<.001) when the 3D-DST was used.ConclusionsThis study demonstrated that the 3D-DST significantly improved the efficiency of delirium assessment and was considered highly acceptable by bedside nurses.

Project description:BackgroundClinical decision support (CDS) is a valuable feature of electronic health records (EHRs) designed to improve quality and safety. However, due to the complexities of system design and inconsistent results, CDS tools may inadvertently increase alert fatigue and contribute to physician burnout. A/B testing, or rapid-cycle randomized tests, is a useful method that can be applied to the EHR in order to rapidly understand and iteratively improve design choices embedded within CDS tools.ObjectiveThis paper describes how rapid randomized controlled trials (RCTs) embedded within EHRs can be used to quickly ascertain the superiority of potential CDS design changes to improve their usability, reduce alert fatigue, and promote quality of care.MethodsA multistep process combining tools from user-centered design, A/B testing, and implementation science was used to understand, ideate, prototype, test, analyze, and improve each candidate CDS. CDS engagement metrics (alert views, acceptance rates) were used to evaluate which CDS version is superior.ResultsTo demonstrate the impact of the process, 2 experiments are highlighted. First, after multiple rounds of usability testing, a revised CDS influenza alert was tested against usual care CDS in a rapid (~6 weeks) RCT. The new alert text resulted in minimal impact on reducing firings per patients per day, but this failure triggered another round of review that identified key technical improvements (ie, removal of dismissal button and firings in procedural areas) that led to a dramatic decrease in firings per patient per day (23.1 to 7.3). In the second experiment, the process was used to test 3 versions (financial, quality, regulatory) of text supporting tobacco cessation alerts as well as 3 supporting images. Based on 3 rounds of RCTs, there was no significant difference in acceptance rates based on the framing of the messages or addition of images.ConclusionsThese experiments support the potential for this new process to rapidly develop, deploy, and rigorously evaluate CDS within an EHR. We also identified important considerations in applying these methods. This approach may be an important tool for improving the impact of and experience with CDS.Trial registrationFlu alert trial: ClinicalTrials.gov NCT03415425; https://clinicaltrials.gov/ct2/show/NCT03415425. Tobacco alert trial: ClinicalTrials.gov NCT03714191; https://clinicaltrials.gov/ct2/show/NCT03714191.

Dataset Information

Usability of a perioperative medication-related clinical decision support software application: a randomized controlled trial.

Objective

Materials and methods

Results

Conclusions

Publications

Usability of a perioperative medication-related clinical decision support software application: a randomized controlled trial.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets