Patient Factors Associated With Biased Language in Nightly Resident Verbal Handoff.
Ontology highlight
ABSTRACT:
SUBMITTER: Wesevich A
PROVIDER: S-EPMC10425858 | biostudies-literature | 2023 Oct
REPOSITORIES: biostudies-literature
Publications
Patient Factors Associated With Biased Language in Nightly Resident Verbal Handoff.
Wesevich Austin A Patel-Nguyen Sonya S Fridman Ilona I Langan Erica E Parente Victoria V
JAMA pediatrics 20231001 10
Similar Datasets
Project description:Objective1) To develop validity evidence for the use of the Verbal Handoff Assessment Tool (VHAT) and examine the reliability of VHAT scores, and 2) to determine whether implementation of a resident handoff bundle (RHB) was associated with improved verbal patient handoffs among pediatric resident physicians.MethodsIn a pre-post design, prospectively audio recorded verbal patient handoffs conducted at Boston Children's Hospital before and after implementation of the RHB were rated using the VHAT, which was developed for this study (primary outcome). Using generalizability theory, we evaluated the reliability of VHAT scores.ResultsOverall, VHAT scores increased after RHB implementation (mean 142 vs 191, possible score 0-500; P < .0001). When accounting for clustering according to resident physician, hospital unit, unit census, and patient complexity, implementation of the RHB was associated with a 63-point increase in VHAT score. Using generalizability theory, we determined that a resident's mean VHAT score on the basis of a handoff of 15 patients assessed by a single observer was sufficiently reliable for relative ranking decisions (ie, norm-based; generalizability coefficient, 0.81), whereas a VHAT score on the basis of a handoff of 21 patients would be sufficiently reliable for high-stakes, standard-based decisions (Phi, 0.80).ConclusionsVerbal handoffs improved after implementation of a RHB, although gains were variable across the 2 clinical units. The VHAT shows promise as an assessment tool for resident handoff skills. If used for competency or entrustment decisions, a resident's mean VHAT score should be on the basis of observation of verbal handoff of ≥21 patients.
| S-EPMC5504880 | biostudies-literature
Project description:BackgroundThe I-PASS Handoff Study found that introduction of a handoff bundle (handoff and teamwork training for residents, a mnemonic, a handoff tool, a faculty development program, and a sustainability campaign) at 9 pediatrics residency programs was associated with improved communication and patient safety.ObjectiveThis parallel qualitative study aimed to understand resident experiences with I-PASS and to inform future implementation and sustainability strategies.MethodsResident experiences with I-PASS were explored in focus groups (N = 50 residents) at 8 hospitals throughout 2012-2013. A content analysis of transcripts was conducted following the principles of grounded theory.ResultsResidents generally accepted I-PASS as an ideal format for handoffs, and valued learning a structured approach. Across all sites, residents reported full adherence to I-PASS when observed, but selective adherence in usual practice. Residents adhered more closely when patients were complex, teams were unfamiliar, and during evening handoff. Residents reported using elements of the I-PASS mnemonic variably, with Illness Severity and Action Items most consistently used, but Synthesis by Receiver least used, except when observed. Most residents were receptive to the electronic handoff tool, but perceptions about usability varied across sites. Experiences with observation and feedback were mixed. Concern about efficiency commonly influenced attitudes about I-PASS.ConclusionsResidents generally supported I-PASS implementation, but adherence was influenced by patient type, context, and individual and team factors. Our findings could inform future implementation, particularly around the areas of resident engagement in change, sensitivity to resident level, perceived efficiency, and faculty observation.
| S-EPMC5476380 | biostudies-literature
Project description:BACKGROUND: Improving handoff communications is a National Patient Safety Goal. Interns and residents are rarely taught how to safely handoff their patients. Our objective was to determine whether teaching safe handoff principles would improve handoff quality. METHODS: Our study was conducted on the inpatient services at 2 teaching hospitals. In this single-institution, randomized controlled trial, internal medicine interns (N = 44) and residents (N = 24) participated in a 45-minute educational session on safe handoff communication skills. Residents received additional education on effective feedback practices and were asked to provide each intern with structured feedback. Quality of interns' electronic and verbal handoffs was measured by using a Handoff Evaluation Tool created by the authors. The frequency of handoff communication failures was also assessed through semistructured phone interviews of postcall interns. RESULTS: Interns who received handoff education demonstrated superior verbal handoff skills than control interns (P < .001), while no difference was seen in electronic handoff skills. Communication failures related to code status (P < .001) and overnight tasks (P < .050) were less frequent in the intervention group. CONCLUSIONS: Interns' electronic handoff documentation skills did not improve with the intervention. This may reflect greater difficulty in changing physicians' electronic documentation habits.
| S-EPMC3399614 | biostudies-other
Project description:ImportanceAn emergency medicine (EM) handoff note generated by a large language model (LLM) has the potential to reduce physician documentation burden without compromising the safety of EM-to-inpatient (IP) handoffs.ObjectiveTo develop LLM-generated EM-to-IP handoff notes and evaluate their accuracy and safety compared with physician-written notes.Design, setting, and participantsThis cohort study used EM patient medical records with acute hospital admissions that occurred in 2023 at NewYork-Presbyterian/Weill Cornell Medical Center. A customized clinical LLM pipeline was trained, tested, and evaluated to generate templated EM-to-IP handoff notes. Using both conventional automated methods (ie, recall-oriented understudy for gisting evaluation [ROUGE], bidirectional encoder representations from transformers score [BERTScore], and source chunking approach for large-scale inconsistency evaluation [SCALE]) and a novel patient safety-focused framework, LLM-generated handoff notes vs physician-written notes were compared. Data were analyzed from October 2023 to March 2024.ExposureLLM-generated EM handoff notes.Main outcomes and measuresLLM-generated handoff notes were evaluated for (1) lexical similarity with respect to physician-written notes using ROUGE and BERTScore; (2) fidelity with respect to source notes using SCALE; and (3) readability, completeness, curation, correctness, usefulness, and implications for patient safety using a novel framework.ResultsIn this study of 1600 EM patient records (832 [52%] female and mean [SD] age of 59.9 [18.9] years), LLM-generated handoff notes, compared with physician-written ones, had higher ROUGE (0.322 vs 0.088), BERTScore (0.859 vs 0.796), and SCALE scores (0.691 vs 0.456), indicating the LLM-generated summaries exhibited greater similarity and more detail. As reviewed by 3 board-certified EM physicians, a subsample of 50 LLM-generated summaries had a mean (SD) usefulness score of 4.04 (0.86) out of 5 (compared with 4.36 [0.71] for physician-written) and mean (SD) patient safety scores of 4.06 (0.86) out of 5 (compared with 4.50 [0.56] for physician-written). None of the LLM-generated summaries were classified as a critical patient safety risk.Conclusions and relevanceIn this cohort study of 1600 EM patient medical records, LLM-generated EM-to-IP handoff notes were determined superior compared with physician-written summaries via conventional automated evaluation methods, but marginally inferior in usefulness and safety via a novel evaluation framework. This study suggests the importance of a physician-in-loop implementation design for this model and demonstrates an effective strategy to measure preimplementation patient safety of LLM models.
| S-EPMC11615705 | biostudies-literature
Project description:BackgroundThis study aimed to identify predictive factors for risky discrepancies in the emergency department (ED) by analyzing patient recalls associated with resident-to-attending radiology report discrepancies (RRDs).ResultsThis retrospective study analyzed 759 RRDs in computed tomography (CT) and magnetic resonance imaging and their outcomes from 2013 to 2021. After excluding 73 patients lost to follow-up, we included 686 records in the final analysis. Risky discrepancies were defined as RRDs resulting in (1) inpatient management (hospitalization) and (2) adverse outcomes (delayed operations, 30-day in-hospital mortality, or intensive care unit admission). Predictors of risky discrepancies were assessed using multivariable logistic regression analysis. The overall RRD rate was 0.4% (759 of 171,419). Of 686 eligible patients, 21.4% (147 of 686) received inpatient management, and 6.0% (41 of 686) experienced adverse outcomes. RRDs with neurological diseases were associated with the highest ED revisit rate (79.4%, 81 of 102) but not with risky RRDs. Predictive factors of inpatient management were critical finding (odds ratio [OR], 5.60; p < 0.001), CT examination (OR, 3.93; p = 0.01), digestive diseases (OR, 2.54; p < 0.001), and late finalized report (OR, 1.65; p = 0.02). Digestive diseases (OR, 6.14; p = 0.006) were identified as the only significant predictor of adverse outcomes.ConclusionsRisky RRDs were associated with several factors, including CT examination, digestive diseases, and late finalized reports, as well as critical image findings. This knowledge could aid in determining the priority of discrepancies for the appropriate management of RRDs.
| S-EPMC9167364 | biostudies-literature
Project description:Objectives: This study tests the null hypotheses that overall sentiment and gendered words in verbal feedback and resident operative autonomy relative to performance are similar for female and male residents. Background: Female and male surgical residents may experience training differently, affecting the quality of learning and graduated autonomy. Methods: A longitudinal, observational study using a Society for Improving Medical Professional Learning collaborative dataset describing resident and attending evaluations of resident operative performance and autonomy and recordings of verbal feedback from attendings from surgical procedures performed at 54 US general surgery residency training programs from 2016 to 2021. Overall sentiment, adjectives, and gendered words in verbal feedback were quantified by natural language processing. Resident operative autonomy and performance, as evaluated by attendings, were reported on 5-point ordinal scales. Performance-adjusted autonomy was calculated as autonomy minus performance. Results: The final dataset included objective assessments and dictated feedback for 2683 surgical procedures. Sentiment scores were higher for female residents (95 [interquartile range (IQR), 4–100] vs 86 [IQR 2–100]; P < 0.001). Gendered words were present in a greater proportion of dictations for female residents (29% vs 25%; P = 0.04) due to male attendings disproportionately using male-associated words in feedback for female residents (28% vs 23%; P = 0.01). Overall, attendings reported that male residents received greater performance-adjusted autonomy compared with female residents (P < 0.001). Conclusions: Sentiment and gendered words in verbal feedback and performance-adjusted operative autonomy differed for female and male general surgery residents. These findings suggest a need to ensure that trainees are given appropriate and equitable operative autonomy and feedback.
| S-EPMC10431433 | biostudies-literature
Project description:Human accuracy in detecting deception with intuitive judgments has been proven to not go above the chance level. Therefore, several automatized verbal lie detection techniques employing Machine Learning and Transformer models have been developed to reach higher levels of accuracy. This study is the first to explore the performance of a Large Language Model, FLAN-T5 (small and base sizes), in a lie-detection classification task in three English-language datasets encompassing personal opinions, autobiographical memories, and future intentions. After performing stylometric analysis to describe linguistic differences in the three datasets, we tested the small- and base-sized FLAN-T5 in three Scenarios using 10-fold cross-validation: one with train and test set coming from the same single dataset, one with train set coming from two datasets and the test set coming from the third remaining dataset, one with train and test set coming from all the three datasets. We reached state-of-the-art results in Scenarios 1 and 3, outperforming previous benchmarks. The results revealed also that model performance depended on model size, with larger models exhibiting higher performance. Furthermore, stylometric analysis was performed to carry out explainability analysis, finding that linguistic features associated with the Cognitive Load framework may influence the model's predictions.
| S-EPMC10739834 | biostudies-literature
Project description:To characterize interactivity during resident and nurse handoffs by investigating listening and question-asking behaviors during conversations. Resident (n = 149) and nurse (n = 126) handoffs in an inpatient medicine unit were audio-recorded. Handoffs were coded based on listening behaviors (active and passive), question types (patient status, coordination of care, clinical reasoning, and framing and alignment), and question responses. Comparisons between residents and nurses for listening and question-asking behaviors were performed using the Wilcoxon rank-sum tests. A Poisson regression model was used to investigate differences in the question-asking behaviors between residents and nurses, and the association between listening and question-asking behaviors. There were no significant differences between residents and nurses in their active (18% resident vs 39% nurse handoffs) or passive (88% resident vs 81% nurse handoffs) listening behaviors. Question-asking was common in resident and nurse handoffs (87% vs 98%) and focused primarily on patient status, co-ordination, and framing and alignment. Nurses asked significantly more questions than residents (Mresident = 2.06 and Mnurse = 5.52) by a factor of 1.76 (P < 0.001). Unit increase in listening behaviors was associated with an increase in the number of questions during resident and nurse handoffs by 7% and 12%, respectively. As suggested by the Joint Commission, question-asking behaviors were common across resident and nurse handoffs, playing a critical role in supporting resilience in communication and collaborative cross-checks during conversations. The role of listening in initiating question-asking behaviors is discussed.
| S-EPMC7309249 | biostudies-literature
Project description:BackgroundDespite the identification of transfer of patient responsibility as a Core Entrustable Professional Activity for Entering Residency, rigorous methods to evaluate incoming residents' ability to give a verbal handoff of multiple patients are lacking.AimOur purpose was to implement a multi-patient, simulation-based curriculum to assess verbal handoff performance.SettingGraduate Medical Education (GME) orientation at an urban, academic medical center.ParticipantsEighty-four incoming residents from four residency programs participated in the study.Program descriptionThe curriculum featured an online training module and a multi-patient observed simulated handoff experience (M-OSHE). Participants verbally "handed off" three mock patients of varying acuity and were evaluated by a trained "receiver" using an expert-informed, five-item checklist.Program evaluationPrior handoff experience in medical school was associated with higher checklist scores (23% none vs. 33% either third OR fourth year vs. 58% third AND fourth year, p = 0.021). Prior training was associated with prioritization of patients based on acuity (12% no training vs. 38% prior training, p = 0.014). All participants agreed that the M-OSHE realistically portrayed a clinical setting.ConclusionsThe M-OSHE is a promising strategy for teaching and evaluating entering residents' ability to give verbal handoffs of multiple patients. Prior training and more handoff experience was associated with higher performance, which suggests that additional handoff training in medical school may be of benefit.
| S-EPMC4803693 | biostudies-literature
Project description:ImportanceBiased patient behavior negatively impacts resident well-being. Data on the prevalence and frequency of these encounters are lacking and are needed to guide the creation of institutional trainings and policies to support trainees.ObjectiveTo evaluate the frequency of resident experiences with and responses to a range of biased patient behaviors.Design, setting, and participantsA retrospective survey was sent via email to 331 second- and third-year internal medicine residents from 3 academic medical centers in California and North Carolina. First-year residents were excluded owing to their limited interactions with patients at the time of participant recruitment. Data were collected from August 21 to November 25, 2019.Main outcomes and measuresDescriptive statistics were used to report the frequency of experience of various types of biased patient behavior, residents' responses, the factors impeding residents' responses, and residents' experiences and beliefs regarding training and policies.ResultsOverall, 232 of 331 residents (70%) participated; 116 (50%) were women; 116 of 247 (47%) were White (participants had the option of selecting >1 race/ethnicity); and 23 (10%) identified as lesbian, gay, bisexual, transgender, or queer. The frequency of resident-reported experience of types of biased patient behaviors varied. The most common behaviors-belittling comments and assumption of nonphysician status-were reported to be experienced 1 or more times per week by 14% of residents (32 of 231) and 17% of residents (38 of 230), respectively. Women, Black or Latinx, and Asian residents reported experiencing biased behavior more frequently. Forty-five percent of Black or Latinx residents (17 of 38) encountered instances of explicit epithets or rejection of care. All 70 Asian residents reported experiencing inquiries into their ethnic origins. Most women residents (110 of 115 [96%]) experienced role questioning behaviors, and 87% (100 of 115) experienced sexual harassment. The need to prioritize clinical care and a sense of futility in responding were the most common factors (cited by 34% of residents [76 of 227] and 25% of residents [56 of 227], respectively) significantly impeding responses to biased behaviors. Eighty-five percent of residents (191 of 226) never reported incidents to their institution. Eighty-nine percent of residents (206 of 232) identified training and policies as necessary or very necessary.Conclusions and relevanceThis survey study suggests that biased patient behavior is experienced frequently by internal medicine residents. Non-White and women residents reported experiencing a disproportionate burden of these incidents. Residents' responses rarely included institutional involvement. Residency programs and health care systems should prioritize training and policies to address biased patient behavior and support affected residents.
| S-EPMC7684448 | biostudies-literature