Dataset Information

The integrative feedback tool: assessing a novel feedback tool among emergency medicine residents.

ABSTRACT:

Objective

Feedback is critical to the growth of learners. However, feedback quality can be variable in practice. Most feedback tools are generic, with few targeting emergency medicine. We created a feedback tool designed for emergency medicine residents, and this study aimed to evaluate the effectiveness of this tool.

Methods

This was a single-center, prospective cohort study comparing feedback quality before and after introducing a novel feedback tool. Residents and faculty completed a survey after each shift assessing feedback quality, feedback time, and the number of feedback episodes. Feedback quality was assessed using a composite score from seven questions, which were each scored 1 to 5 points (minimum total score, 7 points; maximum, 35 points). Preintervention and postintervention data were analyzed using a mixed-effects model that took into account the correlation of random effects between study participants.

Results

Residents completed 182 surveys and faculty members completed 158 surveys. The use of the tool was associated with improved consistency in the summative score of effective feedback attributes as assessed by residents (P=0.040) but not by faculty (P=0.259). However, most of the individual scores for attributes of good feedback did not reach statistical significance. With the tool, residents perceived that faculty spent more time providing feedback (P=0.040) and that the delivery of feedback was more ongoing throughout the shift (P=0.020). Faculty felt that the tool allowed for more ongoing feedback (P=0.002), with no perceived increase in the time spent delivering feedback (P=0.833).

Conclusion

The use of a dedicated tool may help educators provide more meaningful and frequent feedback without impacting the perceived required time needed to provide feedback.

SUBMITTER: Gore KM

PROVIDER: S-EPMC10579731 | biostudies-literature | 2023 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

The integrative feedback tool: assessing a novel feedback tool among emergency medicine residents.

Gore Katarzyna M KM Schiebout Jessen J Peksa Gary D GD Hock Sara S Patwari Rahul R Gottlieb Michael M

Clinical and experimental emergency medicine 20230216 3

<h4>Objective</h4>Feedback is critical to the growth of learners. However, feedback quality can be variable in practice. Most feedback tools are generic, with few targeting emergency medicine. We created a feedback tool designed for emergency medicine residents, and this study aimed to evaluate the effectiveness of this tool.<h4>Methods</h4>This was a single-center, prospective cohort study comparing feedback quality before and after introducing a novel feedback tool. Residents and faculty compl ...[more]

PMID: 36796780

Similar Datasets

Project description:BackgroundIn 2013 the Accreditation Council for Graduate Medical Education (ACGME) introduced "Milestones" designed to nationally standardize the assessment of resident physicians. Previous studies compare resident self-assessment on milestones to faculty assessment, with varying degrees of agreement, but integration of self-assessment into the formative feedback process has not yet been directly studied. This study uses a conceptual framework of self-determination theory, integrated with concepts from adult learning theory, to compare the perception of the feedback quality given in semiannual reviews before and after the incorporation of resident self-assessment into the feedback process.MethodsThis was an interventional study conducted in a single emergency medicine residency program at a major academic hospital over 1 calendar year. Residents first engaged in a semiannual review without self-assessment. At subsequent semiannual reviews, residents completed a Milestone-based self-assessment that was provided to the faculty member assigned to conduct their semiannual review. Residents and faculty completed surveys rating perception of feedback quality. Two-sided Wilcoxon signed-rank tests were used in comparison analysis.ResultsOne resident did not self-assess prior to the semiannual review and was excluded leaving 25 paired surveys for analysis. Residents found feedback after the self-assessment more actionable (p = 0.013), insightful (p = 0.010), and better overall (p = 0.025). Similarly, faculty felt the feedback they provided was more actionable (p < 0.001), more insightful (p < 0.001), and better communicated (p < 0.001); led to improved resident understanding of milestones (p < 0.001); and were overall more satisfied (p < 0.001). Free-text comments explore pre- and postintervention perceptions of feedback.ConclusionsIntegration of self-assessment into semiannual reviews improves perception of feedback given to residents as perceived by both residents and faculty. Although limited by sample size, the results are promising for a simple, evidence-based intervention to improve feedback during an existing mandated feedback opportunity.

Project description:BackgroundThe Accreditation Council for Graduate Medical Education (ACGME) emphasizes constructive feedback as a critical component of residency training. Despite over a decade of using competency-based milestone evaluations, emergency medicine (EM) residency programs lack a standardized method for assessing the quality of feedback. We developed two novel EM-specific feedback surveys to assess the quality of feedback in the ER (FEED-ER) from both the resident and the faculty perspectives. This study aimed to evaluate the surveys' psychometric properties.MethodsWe developed FEED-ER using a Likert scale with faculty and resident versions based on the ACGME framework and a literature review. The preliminary survey consisted of 25 questions involving the feedback domains of timeliness, respect/communication, specificity, action plan, and feedback culture. We conducted two modified Delphi rounds involving 17 content experts to ensure respondent understanding of the items, item coherence to corresponding feedback domains, thematic saturation of domain content, and time duration. A multicenter study was conducted at five university-based EDs in the United States and one in Thailand in 2019. We evaluated the descriptive statistics of the frequency of responses, validity evidence, and reliability of FEED-ER.ResultsA total of 147 EM faculty and 126 EM residents completed the survey. Internal consistency was adequate (Cronbach's alpha > 0.70) and test-retest reliability showed adequate temporal stability (ICC > 0.80) for all dimensions. Content validity was deemed acceptable (CVC > 0.80) for all items. From the 25 items of FEED-ER, 23 loaded into the originally theorized dimensions (with factor loadings > 0.50). Additionally, the five feedback domains were found to be statistically distinct, with correlations between 0.40 and 0.60. The final survey has 23 items.ConclusionsThis is the first study to develop and provide validity evidence for an EM-specific feedback tool that has strong psychometric properties, is reproducible and reliable, and provides an objective measure for assessing the quality of feedback in the ED.

Project description:ObjectivesResidents in emergency medicine have reported dissatisfaction with feedback. One strategy to improve feedback is to enhance learners' feedback literacy-i.e., capabilities as seekers, processors, and users of performance information. To do this, however, the context in which feedback occurs needs to be understood. We investigated how residents typically engage with feedback in an emergency department, along with the potential opportunities to improve feedback engagement in this context. We used this information to develop a program to improve learners' feedback literacy in context and traced the reported translation to practice.MethodsWe conducted a year-long design-based research study informed by agentic feedback principles. Over five cycles in 2019, we interviewed residents and iteratively developed a feedback literacy program. Sixty-six residents participated and data collected included qualitative evaluation surveys (n = 55), educator-written reflections (n = 5), and semistructured interviews with residents (n = 21). Qualitative data were analyzed using framework analysis.ResultsWhen adopting an agentic stance, residents reported changes to the frequency and tenor of their feedback conversations, rendering the interactions more helpful. Despite reporting overall shifts in their conceptions of feedback, they needed to adjust their feedback engagement depending on changing contextual factors such as workload. These microsocial adjustments suggest their feedback literacy develops through an interdependent process of individual intention for feedback engagement-informed by an agentic stance-and dynamic adjustment in response to the environment.ConclusionsResident feedback literacy is profoundly contextualized, so developing feedback literacy in emergency contexts is more nuanced than previously reported. While feedback literacy can be supported through targeted education, our findings raise questions for understanding how emergency medicine environments afford and constrain learner feedback engagement. Our findings also challenge the extent to which this contextual feedback know-how can be "developed" purposefully outside of the everyday work.

Project description:Audience and type of curriculumThis longitudinal leadership curriculum is designed for emergency medicine residents at all levels, with individual sessions designed for each residency year.Length of curriculumThis curriculum runs once annually over three to four years of emergency medicine residency.IntroductionLeadership is a vital skill for emergency physicians but is often passively taught during residency training. Strong leadership skills can lead to improved patient outcomes, but very few residency programs in any specialty and no emergency medicine residency programs have published comprehensive leadership training curricula.Educational goalsThe goals of this curriculum are to expose Emergency Medicine residents to the basics of leadership, to provide a graduated series of interactive, psychologically safe environments to explore individual leadership styles, to review interesting relevant literature, and to discuss leadership principles and experiences with senior leaders in our Emergency Department.Educational methodsThe educational strategies used in this curriculum include: brief lecture-style seminars, small group discussion and reflection, and a panel-style discussion.Research methodsThe educational content of this curriculum was evaluated by learners via feedback surveys after each session.ResultsCourse evaluations conducted in both 2017 and 2020 showed that more than 89% of resident participants found these sessions "useful" or "very useful." All residents surveyed agreed that leadership is an important topic for emergency medicine residency, and 76% felt that the inclusion of leadership content strengthened the residency's curriculum. Suggestions for future topics included handling personal conflict and discussing transitions in leadership during yearly residency promotions.DiscussionThe curriculum has been successfully implemented for seven years. It has proven to be sustainable and requires minimal resources. The residents report high satisfaction with the curriculum and agree that formal instruction on the topic of leadership is important to their on-shift performance and careers.TopicsLeadership, communication.

Project description:BackgroundFeedback is critical for physician development. Multisource feedback is especially important in a team-based specialty such as emergency medicine (EM) and is required by the Accreditation Council for Graduate Medical Education. Nursing assessments provide a unique perspective, but little is known about the current national patterns of their collection and use in EM.MethodsWe surveyed EM program directors using a mixed-methods approach to explore the use of nursing assessment of EM residents. Descriptive data were reported as absolute numbers and percentages. An adjunct analysis of free-text responses was done using the framework method.ResultsThe response rate for our survey was 63% (190 responses), of which 84% currently collect nursing feedback. Respondents from 94% of programs agreed that nursing feedback is useful in assessing professionalism and respondents from 92% of programs agreed that nursing feedback is useful in assessing communication and interpersonal skills, while 44% agreed that it is useful in informing resident medical knowledge. Forty-two percent reported that nursing feedback did not directly influence residents' progression through their training, while 2% indicated that such feedback played a significant role in leading to dismissal or probation. The majority of programs (64%) that do not collect feedback from nurses have done so in the past and hope to do so in the future. Qualitative analysis revealed themes of logistic challenges with data collection, concern regarding quality of feedback, and retributive or gender-disparate feedback.ConclusionsNursing assessments of EM residents were collected by most responding programs and majority of those who do not collect them presently wish to do so in the future. They were considered particularly useful in the assessment of interpersonal skills, communication, and professionalism. However, lack of uniform methods for collecting assessment that meaningfully informs resident development and progression represents a challenge and direction for future inquiry.

Dataset Information

The integrative feedback tool: assessing a novel feedback tool among emergency medicine residents.

Objective

Methods

Results

Conclusion

Publications

The integrative feedback tool: assessing a novel feedback tool among emergency medicine residents.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets