Dataset Information

Diagnostic accuracy and usability of the EMBalance decision support system for vestibular disorders in primary care: proof of concept randomised controlled study results.

ABSTRACT:

Background

Dizziness and imbalance are common symptoms that are often inadequately diagnosed or managed, due to a lack of dedicated specialists. Decision Support Systems (DSS) may support first-line physicians to diagnose and manage these patients based on personalised data.

Aim

To examine the diagnostic accuracy and application of the EMBalance DSS for diagnosis and management of common vestibular disorders in primary care.

Methods

Patients with persistent dizziness were recruited from primary care in Germany, Greece, Belgium and the UK and randomised to primary care clinicians assessing the patients with (+ DSS) versus assessment without (- DSS) the EMBalance DSS. Subsequently, specialists in neuro-otology/audiovestibular medicine performed clinical evaluation of each patient in a blinded way to provide the "gold standard" against which the + DSS, - DSS and the DSS as a standalone tool (i.e. without the final decision made by the clinician) were validated.

Results

One hundred ninety-four participants (age range 25-85, mean = 57.7, SD = 16.7 years) were assigned to the + DSS (N = 100) and to the - DSS group (N = 94). The diagnosis suggested by the + DSS primary care physician agreed with the expert diagnosis in 54%, compared to 41.5% of cases in the - DSS group (odds ratio 1.35). Similar positive trends were observed for management and further referral in the + DSS vs. the - DSS group. The standalone DSS had better diagnostic and management accuracy than the + DSS group.

Conclusion

There were trends for improved vestibular diagnosis and management when using the EMBalance DSS. The tool requires further development to improve its diagnostic accuracy, but holds promise for timely and effective diagnosis and management of dizzy patients in primary care.

Trial registration number

NCT02704819 (clinicaltrials.gov).

SUBMITTER: Bamiou DE

PROVIDER: S-EPMC8527447 | biostudies-literature | 2022 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Diagnostic accuracy and usability of the EMBalance decision support system for vestibular disorders in primary care: proof of concept randomised controlled study results.

Bamiou Doris-Eva DE Kikidis Dimitris D Bibas Thanos T Koohi Nehzat N Macdonald Nora N Maurer Christoph C Wuyts Floris L FL Ihtijarevic Berina B Celis Laura L Mucci Viviana V Maes Leen L Van Rompaey Vincent V Van de Heyning Paul P Nazareth Irwin I Exarchos Themis P TP Fotiadis Dimitrios D Koutsouris Dimitrios D Luxon Linda M LM

Journal of neurology 20211020 5

<h4>Background</h4>Dizziness and imbalance are common symptoms that are often inadequately diagnosed or managed, due to a lack of dedicated specialists. Decision Support Systems (DSS) may support first-line physicians to diagnose and manage these patients based on personalised data.<h4>Aim</h4>To examine the diagnostic accuracy and application of the EMBalance DSS for diagnosis and management of common vestibular disorders in primary care.<h4>Methods</h4>Patients with persistent dizziness were r ...[more]

PMID: 34669009

Similar Datasets

Project description:BackgroundAn increasing number of diagnostic decision support systems (DDSS) exist to support patients and physicians in establishing the correct diagnosis as early as possible. However, little evidence exists that supports the effectiveness of these DDSS. The objectives were to compare the diagnostic accuracy of medical students, with and without the use of a DDSS, and the diagnostic accuracy of the DDSS system itself, regarding the typical rheumatic diseases and to analyze the user experience.MethodsA total of 102 medical students were openly recruited from a university hospital and randomized (unblinded) to a control group (CG) and an intervention group (IG) that used a DDSS (Ada - Your Health Guide) to create an ordered diagnostic hypotheses list for three rheumatic case vignettes. Diagnostic accuracy, measured as the presence of the correct diagnosis first or at all on the hypothesis list, was the main outcome measure and evaluated for CG, IG, and DDSS.ResultsThe correct diagnosis was ranked first (or was present at all) in CG, IG, and DDSS in 37% (40%), 47% (55%), and 29% (43%) for the first case; 87% (94%), 84% (100%), and 51% (98%) in the second case; and 35% (59%), 20% (51%), and 4% (51%) in the third case, respectively. No significant benefit of using the DDDS could be observed. In a substantial number of situations, the mean probabilities reported by the DDSS for incorrect diagnoses were actually higher than for correct diagnoses, and students accepted false DDSS diagnostic suggestions. DDSS symptom entry greatly varied and was often incomplete or false. No significant correlation between the number of symptoms extracted and diagnostic accuracy was seen. It took on average 7 min longer to solve a case using the DDSS. In IG, 61% of students compared to 90% in CG stated that they could imagine using the DDSS in their future clinical work life.ConclusionsThe diagnostic accuracy of medical students was superior to the DDSS, and its usage did not significantly improve students' diagnostic accuracy. DDSS usage was time-consuming and may be misleading due to prompting wrong diagnoses and probabilities.Trial registrationDRKS.de, DRKS00024433 . Retrospectively registered on February 5, 2021.

Project description:BackgroundComputerized clinical decision support systems (CDSSs) are increasingly adopted in health care to optimize resources and streamline patient flow. However, they often lack scientific validation against standard medical care.ObjectiveThe purpose of this study was to assess the performance, safety, and usability of a CDSS in a university hospital emergency department setting in Kuopio, Finland.MethodsPatients entering the emergency department were asked to voluntarily participate in this study. Patients aged 17 years or younger, patients with cognitive impairments, and patients who entered the unit in an ambulance or with the need for immediate care were excluded. Patients completed the CDSS web-based form and usability questionnaire when waiting for the triage nurse's evaluation. The CDSS data were anonymized and did not affect the patients' usual evaluation or treatment. Retrospectively, 2 medical doctors evaluated the urgency of each patient's condition by using the triage nurse's information, and urgent and nonurgent groups were created. The International Statistical Classification of Diseases, Tenth Revision diagnoses were collected from the electronic health records. Usability was assessed by using a positive version of the System Usability Scale questionnaire.ResultsIn total, our analyses included 248 patients. Regarding urgency, the mean sensitivities were 85% and 19%, respectively, for urgent and nonurgent cases when assessing the performance of CDSS evaluations in comparison to that of physicians. The mean sensitivities were 85% and 35%, respectively, when comparing the evaluations between the two physicians. Our CDSS did not miss any cases that were evaluated to be emergencies by physicians; thus, all emergency cases evaluated by physicians were evaluated as either urgent cases or emergency cases by the CDSS. In differential diagnosis, the CDSS had an exact match accuracy of 45.5% (97/213). The usability was good, with a mean System Usability Scale score of 78.2 (SD 16.8).ConclusionsIn a university hospital emergency department setting with a large real-world population, our CDSS was found to be equally as sensitive in urgent patient cases as physicians and was found to have an acceptable differential diagnosis accuracy, with good usability. These results suggest that this CDSS can be safely assessed further in a real-world setting. A CDSS could accelerate triage by providing patient-provided data in advance of patients' initial consultations and categorize patient cases as urgent and nonurgent cases upon patients' arrival to the emergency department.

Project description:BackgroundDiagnostic errors are significant problems in medical care. Despite the usefulness of artificial intelligence (AI)-based diagnostic decision support systems, the overreliance of physicians on AI-generated diagnoses may lead to diagnostic errors.ObjectiveWe investigated the safe use of AI-based diagnostic decision support systems with trust calibration by adjusting trust levels to match the actual reliability of AI.MethodsA quasi-experimental study was conducted at Dokkyo Medical University, Japan, with physicians allocated (1:1) to the intervention and control groups. A total of 20 clinical cases were created based on the medical histories recorded by an AI-driven automated medical history-taking system from actual patients who visited a community-based hospital in Japan. The participants reviewed the medical histories of 20 clinical cases generated by an AI-driven automated medical history-taking system with an AI-generated list of 10 differential diagnoses and provided 1 to 3 possible diagnoses. Physicians were asked whether the final diagnosis was in the AI-generated list of 10 differential diagnoses in the intervention group, which served as the trust calibration. We analyzed the diagnostic accuracy of physicians and the correctness of the trust calibration in the intervention group. We also investigated the relationship between the accuracy of the trust calibration and the diagnostic accuracy of physicians, and the physicians' confidence level regarding the use of AI.ResultsAmong the 20 physicians assigned to the intervention (n=10) and control (n=10) groups, the mean age was 30.9 (SD 3.9) years and 31.7 (SD 4.2) years, the proportion of men was 80% and 60%, and the mean postgraduate year was 5.8 (SD 2.9) and 7.2 (SD 4.6), respectively, with no significant differences. The physicians' diagnostic accuracy was 41.5% in the intervention group and 46% in the control group, with no significant difference (95% CI -0.75 to 2.55; P=.27). The overall accuracy of the trust calibration was only 61.5%, and despite correct calibration, the diagnostic accuracy was 54.5%. In the multivariate logistic regression model, the accuracy of the trust calibration was a significant contributor to the diagnostic accuracy of physicians (adjusted odds ratio 5.90, 95% CI 2.93-12.46; P<.001). The mean confidence level for AI was 72.5% in the intervention group and 45% in the control group, with no significant difference.ConclusionsTrust calibration did not significantly improve physicians' diagnostic accuracy when considering the differential diagnoses generated by reading medical histories and the possible differential diagnosis lists of an AI-driven automated medical history-taking system. As this was a formative study, the small sample size and suboptimal trust calibration methods may have contributed to the lack of significant differences. This study highlights the need for a larger sample size and the implementation of supportive measures of trust calibration.

Project description:BackgroundAphasia is the loss or impairment of language functions and affects everyday social life. The disorder leads to the inability to understand and be understood in both written and verbal communication and affects the linguistic modalities of auditory comprehension, verbal expression, reading, and writing. Due to heterogeneity of the impairment, therapy must be adapted individually and dynamically to patient needs. An important factor for successful aphasia therapy is dose and intensity of therapy. Tablet computer-based apps are a promising treatment method that allows patients to train independently at home, is well accepted, and is known to be beneficial for patients. In addition, it has been shown to ease the burden of therapists.ObjectiveThe aim of this project was to develop an adaptive multimodal system that enables aphasic patients to train at home using language-related tasks autonomously, allows therapists to remotely assign individualized tasks in an easy and time-efficient manner, and tracks the patient's progress as well as creation of new individual exercises.MethodsThe system consists of two main parts: (1) the patient's interface, which allows the patient to exercise, and (2) the therapist's interface, which allows the therapist to assign new exercises to the patient and supervise the patient's progress. The pool of exercises is based on a hierarchical language structure. Using questionnaires, therapists and patients evaluated the system in terms of usability (ie, System Usability Scale) and motivation (ie, adapted Intrinsic Motivation Inventory).ResultsA total of 11 speech and language therapists (age: mean 28, SD 7 years) and 15 patients (age: mean 53, SD 10 years) diagnosed with aphasia participated in this study. Patients rated the Bern Aphasia App in terms of usability (scale 0-100) as excellent (score >70; Z=-1.90; P=.03) and therapists rated the app as good (score >85; Z=-1.75; P=.04). Furthermore, patients enjoyed (scale 0-6) solving the exercises (score>3; mean 3.5, SD 0.40; Z=-1.66; P=.049).ConclusionsBased on the questionnaire scores, the system is well accepted and simple to use for patients and therapists. Furthermore, the new tablet computer-based app and the hierarchical language exercise structure allow patients with different types of aphasia to train with different doses and intensities independently at home. Thus, the novel system has potential for treatment of patients with aphasia as a supplement to face-to-face therapy.

Dataset Information

Diagnostic accuracy and usability of the EMBalance decision support system for vestibular disorders in primary care: proof of concept randomised controlled study results.

Background

Aim

Methods

Results

Conclusion

Trial registration number

Publications

Diagnostic accuracy and usability of the EMBalance decision support system for vestibular disorders in primary care: proof of concept randomised controlled study results.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets