Dataset Information

Implementation of a Web-Based Tool With Text Message Prompts to Improve End-of-Shift Assessments for Emergency Medicine Residents.

ABSTRACT:

Background

End-of-shift assessments (ESA) can provide representative data on medical trainee performance but do not occur routinely and are not documented systematically.

Objective

To evaluate the implementation of a web-based tool with text message prompts to assist mobile ESA (mESA) in an emergency medicine (EM) residency program.

Methods

mESA used timed text messages to prompt faculty/trainees to expect in-person qualitative ESA in a milestone content area and for the faculty to record descriptive performance data through a web-based platform. We assessed implementation between January 2018 and November 2019 using the RE-AIM framework (reach, effectiveness, adoption, implementation, and maintenance).

Results

Reach: 96 faculty and 79 trainees participated in the mESA program. Effectiveness: From surveys, approximately 72% of faculty and 58% of trainees reported increases in providing and receiving ESA feedback after program implementation. From ESA submissions, trainees reported receiving in-person feedback on 90% of shifts. Residency leadership confirmed perceived utility of the mESA program. Adoption: mESA prompts were sent on 7792 unique shifts across 4 EDs, all days of week, and different times of day. Faculty electronically submitted ESA feedback on 45% of shifts. Implementation quality: No technological errors occurred. Maintenance: Completion of in-person ESA feedback and electronic submission of feedback by faculty was stable over time.

Conclusions

We found mixed evidence in support of using a web-based tool with text message prompts for mESA for EM trainees.

SUBMITTER: Tobias A

PROVIDER: S-EPMC7771609 | biostudies-literature | 2020 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Implementation of a Web-Based Tool With Text Message Prompts to Improve End-of-Shift Assessments for Emergency Medicine Residents.

Tobias Adam A Sobehart Robert R Doshi Ankur A AA Suffoletto Brian B

Journal of graduate medical education 20201204 6

<h4>Background</h4>End-of-shift assessments (ESA) can provide representative data on medical trainee performance but do not occur routinely and are not documented systematically.<h4>Objective</h4>To evaluate the implementation of a web-based tool with text message prompts to assist mobile ESA (mESA) in an emergency medicine (EM) residency program.<h4>Methods</h4>mESA used timed text messages to prompt faculty/trainees to expect in-person qualitative ESA in a milestone content area and for the fa ...[more]

PMID: 33391600

Similar Datasets

Project description:ImportancePrevious studies have demonstrated sex-specific disparities in performance assessments among emergency medicine (EM) residents. However, less work has focused on intersectional disparities by ethnoracial identity and sex in resident performance assessments.ObjectiveTo estimate intersectional sex-specific ethnoracial disparities in standardized EM resident assessments.Design, setting, and participantsThis retrospective cohort study used data from the Association of American Medical Colleges and the Accreditation Council for Graduate Medical Education Milestones (Milestones) assessments to evaluate ratings for EM residents at 128 EM training programs in the US. Statistical analyses were conducted in June 2020 to January 2023.ExposureTraining and assessment environments in EM residency programs across comparison groups defined by ethnoracial identity (Asian, White, or groups underrepresented in medicine [URM], ie, African American/Black, American Indian/Alaska Native, Hispanic/Latine, and Native Hawaiian/Other Pacific Islander) and sex (female/male).Main outcomes and measuresMean Milestone scores (scale, 0-9) across 6 core competency domains: interpersonal and communications skills, medical knowledge, patient care, practice-based learning and improvement, professionalism, and system-based practice. Overall assessment scores were calculated as the mean of the 6 competency scores.ResultsThe study sample comprised 128 ACGME-accredited programs and 16 634 assessments for 2708 EM residents of which 1913 (70.6%) were in 3-year and 795 (29.4%) in 4-year programs. Most of the residents were White (n = 2012; 74.3%), followed by Asian (n = 477; 17.6%), Hispanic or Latine (n = 213; 7.9%), African American or Black (n = 160; 5.9%), American Indian or Alaska Native (n = 24; 0.9%), and Native Hawaiian or Other Pacific Islander (n = 4; 0.1%). Approximately 14.3% (n = 386) and 34.6% (n = 936) were of URM groups and female, respectively. Compared with White male residents, URM female residents in 3-year programs were rated increasingly lower in the medical knowledge (URM female score, -0.47; 95% CI, -0.77 to -0.17), patient care (-0.18; 95% CI, -0.35 to -0.01), and practice-based learning and improvement (-0.37; 95% CI, -0.65 to -0.09) domains by postgraduate year 3 year-end assessment; URM female residents in 4-year programs were also rated lower in all 6 competencies over the assessment period.Conclusions and relevanceThis retrospective cohort study found that URM female residents were consistently rated lower than White male residents on Milestone assessments, findings that may reflect intersectional discrimination in physician competency evaluation. Eliminating sex-specific ethnoracial disparities in resident assessments may contribute to equitable health care by removing barriers to retention and promotion of underrepresented and minoritized trainees and facilitating diversity and representation among the emergency physician workforce.

Project description:BackgroundUse of a person's name in a text message has been shown to be effective in instigating behaviour change. We evaluated the effectiveness of a personalised text message (including the recipient's name) versus a standardised text message for prompting a response from trial participants to complete and return postal follow-up questionnaires.MethodsUsing a randomised study within a trial (SWAT) embedded within the host GRASP (Getting it Right: Addressing Shoulder Pain) trial, participants who provided a mobile telephone number were randomised (1:1) by a central computer system to receive either (1) a personalised text message which included their name or (2) a standard text message. Text messages were sent by the trial office on the same day as the 6-month GRASP follow-up questionnaire. The primary outcome was questionnaire response rate, defined as the proportion of 6-month GRASP follow-up questionnaires returned by participants. Secondary outcomes included time to response, the proportion of participants sent a reminder follow-up questionnaire, and cost.ResultsBetween March 2017 and May 2019 (recruitment period for GRASP trial), 618 participants were randomised to a personalised (n = 309) or standard (n = 309) text message and all were included in the analysis. The overall questionnaire response rate was 87% (n = 537/618); 90% (n = 277/309) of participants responded in the personalised text message group compared to 84% (n = 260/309) in the standard text message group (relative risk (RR) 1.07; 95% CI 1.00 to 1.13). Participants randomised to receive the personalised text message were more likely to return their initial postal questionnaire than those who received the standard text message (n = 185/309; 60% vs. n = 160/309; 52%) (RR 1.16; 95% CI 1.00 to 1.33); this represents an absolute percentage difference between intervention groups of 8%. Post hoc subgroup analysis showed that males under 65 years were the group most likely to return their initial questionnaire if they received a personalised text message.ConclusionOverall, participants who received a personalised text message were more likely to return their questionnaire than those who received the standard text message.Trial registrationGRASP Trial ISRCTN16539266 ; SWAT Repository ID 35.

Project description:IntroductionEmergency physicians are in a unique position to impact both individual and population health needs. Despite this, emergency medicine (EM) residency training lacks formalized education n the social determinants of health (SDoH) and integration of patient social risk and need, which are core components of social EM (SEM). The need for such a SEM-based residency curriculum has been previously recognized; however, there is a gap in the literature related to demonstration and feasibility. In this study we sought to address this need by implementing and evaluating a replicable, multifaceted introductory SEM curriculum for EM residents. This curriculum is designed to increase general awareness related to SEM and to increase ability to identify and intervene upon SDoH in clinical practice.MethodsA taskforce of EM clinician-educators with expertise in SEM developed a 4.5-hour educational curriculum for use during a single, half-day didactic session for EM residents. The curriculum consisted of asynchronous learning via a podcast, four SEM subtopic lecture didactics, guest speakers from the emergency department (ED) social work team and a community outreach partner, and a poverty simulation with interdisciplinary debrief. We obtained pre- and post- intervention surveys.ResultsA total of 35 residents and faculty attended the conference day, with 18 participants completing the immediate post-conference survey and 10 participants completing the two-month delayed, post-conference survey. Post-survey results demonstrated improved awareness of SEM concepts and increased confidence in participants' knowledge of community resources and ability to connect patients to these resources following the curricular intervention (25% pre-conference to 83% post-conference). In addition, post-survey assessment demonstrated significantly heightened awareness and clinical consideration of SDoH among participants (31% pre-conference to 78% post-conference) and increased comfort in identifying social risk in the ED (75% pre-conference to 94% post-conference). Overall, all components of the curriculum were evaluated as meaningful and specifically beneficial for EM training. The ED care coordination, poverty simulation, and the subtopic lectures were rated most meaningful.ConclusionThis pilot curricular integration study demonstrates feasibility and the perceived participant value of incorporating a social EM curriculum into EM residency training.

Project description:BackgroundEngagement with digital health interventions (DHIs) may be regarded as a prerequisite for the intervention to achieve positive health or behavior change outcomes. One method employed to promote engagement is the use of prompts such as emails and text messages. However, little is known about the characteristics of prompts that promote engagement. This study explored the association between the content and delivery mode of prompts and the users' engagement with HeLP-Diabetes (Healthy Living for People with type 2 Diabetes), a DHI that aimed to promote self-management in adults with type 2 diabetes.ObjectiveThe objective of this study was to identify the characteristics of prompts, specifically the content and delivery mode, which were associated with increased engagement.MethodsThis was a mixed-methods study. Email and text message prompts were sent to the registered users of HeLP-Diabetes. Use of the intervention was recorded and examined to identify which email and text message prompts were associated with subsequent visits to the DHI. Characteristics of prompts that were identified as particularly effective or ineffective were explored through think-aloud interviews with the participants.ResultsOf a total of 39 email prompts, 49% (19/39) prompts showed a significant association with subsequent visits to the DHI. However, none of the text message prompts were associated with subsequent visits to the DHI. Furthermore, think-aloud interviews were carried out with 6 experienced participants with type 2 diabetes. The findings suggest that these participants preferred email prompts that were clear, relatively short, and empowering; used nondirective advice; included health professional references; were visually appealing; and contained news and updates.ConclusionsThe findings of this study contribute to the existing evidence supporting the role of email prompts in promoting and maintaining engagement with DHIs. This study described the content of prompts that may be engaging. However, the results should be interpreted with caution, as prompts may be context-specific interventions and the results may not be generalizable across other DHIs or other types of interventions targeting self-management of type 2 diabetes.

Dataset Information

Implementation of a Web-Based Tool With Text Message Prompts to Improve End-of-Shift Assessments for Emergency Medicine Residents.

Background

Objective

Methods

Results

Conclusions

Publications

Implementation of a Web-Based Tool With Text Message Prompts to Improve End-of-Shift Assessments for Emergency Medicine Residents.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets