Project description:Syndromic surveillance detects and monitors individual and population health indicators through sources such as emergency department records. Automated classification of these records can improve outbreak detection speed and diagnosis accuracy. Current syndromic systems rely on hand-coded keyword-based methods to parse written fields and may benefit from the use of modern supervised-learning classifier models. In this paper, we implement two recurrent neural network models based on long short-term memory (LSTM) and gated recurrent unit (GRU) cells and compare them to two traditional bag-of-words classifiers: multinomial naïve Bayes (MNB) and a support vector machine (SVM). The MNB classifier is one of only two machine learning algorithms currently being used for syndromic surveillance. All four models are trained to predict diagnostic code groups as defined by Clinical Classification Software, first to predict from discharge diagnosis, and then from chief complaint fields. The classifiers are trained on 3.6 million de-identified emergency department records from a single United States jurisdiction. We compare performance of these models primarily using the F1 score, and we measure absolute model performance to determine which conditions are the most amenable to surveillance based on chief complaint alone. Using discharge diagnoses, the LSTM classifier performs best, though all models exhibit an F1 score above 96.00. Using chief complaints, the GRU performs best (F1 = 47.38), and MNB with bigrams performs worst (F1 = 39.40). We also note that certain syndrome types are easier to detect than others. For example, chief complaints using the GRU model predicts alcohol-related disorders well (F1 = 78.91) but predicts influenza poorly (F1 = 14.80). In all instances, the RNN models outperformed the bag-of-words classifiers suggesting deep learning models could substantially improve the automatic classification of unstructured text for syndromic surveillance.

Project description:IntroductionElite military trainees are burdened by high numbers of musculoskeletal (MSK) injuries and are a priority military population for injury prevention. This research aims to describe the MSK complaint epidemiology of trainees undertaking special forces (SF) training in the Australian Defence Force (ADF). One barrier to accurate injury surveillance in military populations is that traditional surveillance methods rely on personnel engaging with the military healthcare system to collect injury data. This approach is likely to underestimate the injury burden as it is known that many military personnel, particularly trainees, avoid reporting their injuries because of various motives. Subsequently, the insights from surveillance systems may underestimate the injury burden and limit the ability to inform prevention requirements. This research aims to actively seek MSK complaint information directly from trainees in a sensitive manner to mediate injury-reporting behaviors.Materials and methodsThis descriptive epidemiology study included two consecutive cohorts of ADF SF trainees from 2019 to 2021. Musculoskeletal data items and their respective recording methods were based on international sports injury surveillance guidelines and adapted to a military context. Our case definition encompassed all injuries or physical discomforts as recordable cases. A unit-embedded physiotherapist retrospectively collected MSK complaint data from selection courses and collected prospective data over the training continuum. Data collection processes were external to the military health care system to mediate reporting avoidance and encourage injury reporting. Injury proportions, complaint incidence rates, and incidence rate ratios were calculated and compared between training courses and cohorts.ResultsIn total, 334 MSK complaints were reported by 103 trainees (90.4%), with a complaint incidence rate of 58.9 per 1,000 training weeks (95% CI, 53.0-65.5). Of these MSK complaints, 6.4% (n = 22) resulted in time loss from work. The lumbar spine (20.6%, n = 71) and the knee (18.9%, n = 65) were the most frequently affected body parts. Most of the MSK complaints were reported during selection courses (41.9%), followed by field survival and team tactics (23.0%) and urban operations courses (21.9%). Physical training accounted for 16.5% of complaints. Fast-roping training was associated with more severe MSK complaints.ConclusionsMusculoskeletal complaints are highly prevalent in ADF SF trainees. Complaints are more frequently reported in selection and qualification training courses than in physical training. These activities are priorities for focused research to understand injury circumstances in ADF elite training programs to inform injury prevention strategies. A strength of our study is the data collection methods which have provided greater MSK complaint information than past research; however, much work remains in conducting consistent and accurate surveillance. Another strength is the use of an embedded physiotherapist to overcome injury-reporting avoidance. Embedded health professionals are recommended as continued practice for ongoing surveillance and early intervention.

Project description:ImportanceReduction in emergency department (ED) use is frequently viewed as a potential source for cost savings. One consideration has been to deny payment if the patient's diagnosis upon ED discharge appears to reflect a "nonemergency" condition. This approach does not incorporate other clinical factors such as chief complaint that may inform necessity for ED care.ObjectiveTo determine whether ED presenting complaint and ED discharge diagnosis correspond sufficiently to support use of discharge diagnosis as the basis for policies discouraging ED use.Design, setting, and participantsThe New York University emergency department algorithm has been commonly used to identify nonemergency ED visits. We applied the algorithm to publicly available ED visit data from the 2009 National Hospital Ambulatory Medical Care Survey (NHAMCS) for the purpose of identifying all "primary care-treatable" visits. The 2009 NHAMCS data set contains 34,942 records, each representing a unique ED visit. For each visit with a discharge diagnosis classified as primary care treatable, we identified the chief complaint. To determine whether these chief complaints correspond to nonemergency ED visits, we then examined all ED visits with this same group of chief complaints to ascertain the ED course, final disposition, and discharge diagnoses.Main outcomes and measuresPatient demographics, clinical characteristics, and disposition associated with chief complaints related to nonemergency ED visits.ResultsAlthough only 6.3% (95% CI, 5.8%-6.7%) of visits were determined to have primary care-treatable diagnoses based on discharge diagnosis and our modification of the algorithm, the chief complaints reported for these ED visits with primary care-treatable ED discharge diagnoses were the same chief complaints reported for 88.7% (95% CI, 88.1%-89.4%) of all ED visits. Of these visits, 11.1% (95% CI, 9.3%-13.0%) were identified at ED triage as needing immediate or emergency care; 12.5% (95% CI, 11.8%-14.3%) required hospital admission; and 3.4% (95% CI, 2.5%-4.3%) of admitted patients went directly from the ED to the operating room.Conclusions and relevanceAmong ED visits with the same presenting complaint as those ultimately given a primary care-treatable diagnosis based on ED discharge diagnosis, a substantial proportion required immediate emergency care or hospital admission. The limited concordance between presenting complaints and ED discharge diagnoses suggests that these discharge diagnoses are unable to accurately identify nonemergency ED visits.

Project description:BackgroundPatient complaints are a perennial challenge faced by health care institutions globally, requiring extensive time and effort from health care workers. Despite these efforts, patient dissatisfaction remains high. Recent studies on the use of large language models (LLMs) such as the GPT models developed by OpenAI in the health care sector have shown great promise, with the ability to provide more detailed and empathetic responses as compared to physicians. LLMs could potentially be used in responding to patient complaints to improve patient satisfaction and complaint response time.ObjectiveThis study aims to evaluate the performance of LLMs in addressing patient complaints received by a tertiary health care institution, with the goal of enhancing patient satisfaction.MethodsAnonymized patient complaint emails and associated responses from the patient relations department were obtained. ChatGPT-4.0 (OpenAI, Inc) was provided with the same complaint email and tasked to generate a response. The complaints and the respective responses were uploaded onto a web-based questionnaire. Respondents were asked to rate both responses on a 10-point Likert scale for 4 items: appropriateness, completeness, empathy, and satisfaction. Participants were also asked to choose a preferred response at the end of each scenario.ResultsThere was a total of 188 respondents, of which 115 (61.2%) were health care workers. A majority of the respondents, including both health care and non-health care workers, preferred replies from ChatGPT (n=164, 87.2% to n=183, 97.3%). GPT-4.0 responses were rated higher in all 4 assessed items with all median scores of 8 (IQR 7-9) compared to human responses (appropriateness 5, IQR 3-7; empathy 4, IQR 3-6; quality 5, IQR 3-6; satisfaction 5, IQR 3-6; P<.001) and had higher average word counts as compared to human responses (238 vs 76 words). Regression analyses showed that a higher word count was a statistically significant predictor of higher score in all 4 items, with every 1-word increment resulting in an increase in scores of between 0.015 and 0.019 (all P<.001). However, on subgroup analysis by authorship, this only held true for responses written by patient relations department staff and not those generated by ChatGPT which received consistently high scores irrespective of response length.ConclusionsThis study provides significant evidence supporting the effectiveness of LLMs in resolution of patient complaints. ChatGPT demonstrated superiority in terms of response appropriateness, empathy, quality, and overall satisfaction when compared against actual human responses to patient complaints. Future research can be done to measure the degree of improvement that artificial intelligence generated responses can bring in terms of time savings, cost-effectiveness, patient satisfaction, and stress reduction for the health care system.

Dataset Information

The Baron's complaint.

Publications

The Baron's complaint.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets