Dataset Information

Assessing information system readiness for mitigating malpractice risk through simulation: results of a multi-site study.

ABSTRACT: OBJECTIVE:To develop and test an instrument for assessing a healthcare organization's ability to mitigate malpractice risk through clinical decision support (CDS). MATERIALS AND METHODS:Based on a previously collected malpractice data set, we identified common types of CDS and the number and cost of malpractice cases that might have been prevented through this CDS. We then designed clinical vignettes and questions that test an organization's CDS capabilities through simulation. Seven healthcare organizations completed the simulation. RESULTS:All seven organizations successfully completed the self-assessment. The proportion of potentially preventable indemnity loss for which CDS was available ranged from 16.5% to 73.2%. DISCUSSION:There is a wide range in organizational ability to mitigate malpractice risk through CDS, with many organizations' electronic health records only being able to prevent a small portion of malpractice events seen in a real-world dataset. CONCLUSION:The simulation approach to assessing malpractice risk mitigation through CDS was effective. Organizations should consider using malpractice claims experience to facilitate prioritizing CDS development.

SUBMITTER: Wright A

PROVIDER: S-EPMC6283077 | biostudies-literature | 2015 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Assessing information system readiness for mitigating malpractice risk through simulation: results of a multi-site study.

Wright Adam A Maloney Francine L FL Wien Matthew M Samal Lipika L Emani Srinivas S Zuccotti Gianna G

Journal of the American Medical Informatics Association : JAMIA 20150526 5

<h4>Objective</h4>To develop and test an instrument for assessing a healthcare organization's ability to mitigate malpractice risk through clinical decision support (CDS).<h4>Materials and methods</h4>Based on a previously collected malpractice data set, we identified common types of CDS and the number and cost of malpractice cases that might have been prevented through this CDS. We then designed clinical vignettes and questions that test an organization's CDS capabilities through simulation. Se ...[more]

PMID: 26017230

Similar Datasets

Project description:BackgroundCognitive biases in clinical decision-making significantly contribute to errors in diagnosis and suboptimal patient outcomes. Addressing these biases presents a formidable challenge in the medical field.ObjectiveThis study aimed to explore the role of large language models (LLMs) in mitigating these biases through the use of the multi-agent framework. We simulate the clinical decision-making processes through multi-agent conversation and evaluate its efficacy in improving diagnostic accuracy compared with humans.MethodsA total of 16 published and unpublished case reports where cognitive biases have resulted in misdiagnoses were identified from the literature. In the multi-agent framework, we leveraged GPT-4 (OpenAI) to facilitate interactions among different simulated agents to replicate clinical team dynamics. Each agent was assigned a distinct role: (1) making the final diagnosis after considering the discussions, (2) acting as a devil's advocate to correct confirmation and anchoring biases, (3) serving as a field expert in the required medical subspecialty, (4) facilitating discussions to mitigate premature closure bias, and (5) recording and summarizing findings. We tested varying combinations of these agents within the framework to determine which configuration yielded the highest rate of correct final diagnoses. Each scenario was repeated 5 times for consistency. The accuracy of the initial diagnoses and the final differential diagnoses were evaluated, and comparisons with human-generated answers were made using the Fisher exact test.ResultsA total of 240 responses were evaluated (3 different multi-agent frameworks). The initial diagnosis had an accuracy of 0% (0/80). However, following multi-agent discussions, the accuracy for the top 2 differential diagnoses increased to 76% (61/80) for the best-performing multi-agent framework (Framework 4-C). This was significantly higher compared with the accuracy achieved by human evaluators (odds ratio 3.49; P=.002).ConclusionsThe multi-agent framework demonstrated an ability to re-evaluate and correct misconceptions, even in scenarios with misleading initial investigations. In addition, the LLM-driven, multi-agent conversation framework shows promise in enhancing diagnostic accuracy in diagnostically challenging medical scenarios.

Dataset Information

Assessing information system readiness for mitigating malpractice risk through simulation: results of a multi-site study.

Publications

Assessing information system readiness for mitigating malpractice risk through simulation: results of a multi-site study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets