Dataset Information

Using Learning Analytics to Examine Differences in Assessment Forms From Continuous Versus Episodic Supervisors of Family Medicine Residents.

ABSTRACT:

Background

It is assumed that there is a need for continuity of supervision within competency-based medical education, despite most evidence coming from the undergraduate medical education rather than the graduate medical education (GME) context. This evidence gap must be addressed to justify the time and effort needed to redesign GME programs to support continuity of supervision.

Objective

To examine differences in assessment behaviors of continuous supervisors (CS) versus episodic supervisors (ES), using completed formative assessment forms, FieldNotes, as a proxy.

Methods

The FieldNotes CS- and ES-entered for family medicine residents (N=186) across 3 outpatient teaching sites over 3 academic years (2015-2016, 2016-2017, 2017-2018) were examined using 2-sample proportion z-tests to determine differences on 3 FieldNote elements: competency (Sentinel Habit [SH]), Clinical Domain (CD), and Progress Level (PL).

Results

Sixty-nine percent (6104 of 8909) of total FieldNotes were analyzed. Higher proportions of CS-entered FieldNotes indicated SH3 (Managing patients with best practices), z=-3.631, P<.0001; CD2 (Care of adults), z=-8.659, P<.0001; CD3 (Care of the elderly), z=-4.592, P<.0001; and PL3 (Carry on, got it), z=-4.482, P<.0001. Higher proportions of ES-entered FieldNotes indicated SH7 (Communication skills), z=4.268, P<.0001; SH8 (Helping others learn), z=20.136, P<.0001; CD1 (Doctor-patient relationship/ethics), z=14.888, P<.0001; CD9 (Not applicable), z=7.180, P<.0001; and PL2 (In progress), z=5.117, P<.0001.

Conclusions

The type of supervisory relationship impacts assessment: there is variability in which competencies are paid attention to, which contexts or populations are included, and which progress levels are chosen.

SUBMITTER: Lee ASO

PROVIDER: S-EPMC9580309 | biostudies-literature | 2022 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Using Learning Analytics to Examine Differences in Assessment Forms From Continuous Versus Episodic Supervisors of Family Medicine Residents.

Lee Ann S O ASO Donoff Christopher C Ross Shelley S

Journal of graduate medical education 20221001 5

<h4>Background</h4>It is assumed that there is a need for continuity of supervision within competency-based medical education, despite most evidence coming from the undergraduate medical education rather than the graduate medical education (GME) context. This evidence gap must be addressed to justify the time and effort needed to redesign GME programs to support continuity of supervision.<h4>Objective</h4>To examine differences in assessment behaviors of continuous supervisors (CS) versus episod ...[more]

PMID: 36274777

Similar Datasets

Project description:BackgroundSingle-center studies have reported residents experience barriers to accessing supervising physicians overnight, but no national dataset has described barriers perceived by residents or the association between supervision models and perceived barriers.ObjectiveTo explore residents' perception of barriers to accessing overnight supervision.DesignQuestions about overnight supervision and barriers to accessing it were included on the American College of Physicians Internal Medicine In-Training Examination® (IM-ITE®) Resident Survey in Fall 2017.ParticipantsAll US-based internal medicine residents who completed the 2017 IM-ITE®. Responses from 20,744 residents (84%) were analyzed.Main measuresFor our main outcome, we calculated percentages of responses for eight barriers and tested for association with the presence or absence of nocturnists. For our secondary outcome, we categorized free-text responses enumerating barriers from all residents into the five Systems Engineering Initiative for Patient Safety (SEIPS) categories to elucidate future areas for study or intervention.Key resultsInternal medicine residents working in hospitals without nocturnists more commonly reported having at least one barrier to accessing a supervising physician "always" or "most of the time" (5075/9842, 51.6%) compared to residents in hospitals with nocturnists (3074/10,902, 28.2%, p < 0.001). Among residents in hospitals without nocturnists, the most frequently reported barrier to accessing attending supervision was attendings not being present in the hospital (30.4% "always" or "most of the time"); residents in hospitals with nocturnists most frequently reported desire to make their own decisions as a barrier to contacting attendings (15.7% "always" or "most of the time"). Free-text responses from residents with and without nocturnists most commonly revealed organization (47%) barriers to accessing supervision; 28% cited person barriers, and 23% cited tools/technology barriers.ConclusionsPresence of nocturnists is associated with fewer reported barriers to contacting supervising physicians overnight. Organizational culture, work schedules, desire for independence, interpersonal interactions, and technology may present important barriers.

Project description:ImportancePrior studies have revealed gender differences in the milestone and clinical competency committee assessment of emergency medicine (EM) residents.ObjectiveTo explore gender disparities and the reasons for such disparities in the narrative comments from EM attending physicians to EM residents.Design, setting, and participantsThis multicenter qualitative analysis examined 10 488 narrative comments among EM faculty and EM residents between 2015 to 2018 in 5 EM training programs in the US. Data were analyzed from 2019 to 2021.Main outcomes and measuresDifferences in narrative comments by gender and study site. Qualitative analysis included deidentification and iterative coding of the data set using an axial coding approach, with double coding of 20% of the comments at random to assess intercoder reliability (κ, 0.84). The authors reviewed the unmasked coded data set to identify emerging themes. Summary statistics were calculated for the number of narrative comments and their coded themes by gender and study site. χ2 tests were used to determine differences in the proportion of narrative comments by gender of faculty and resident.ResultsIn this study of 283 EM residents, of whom 113 (40%) identified as women, and 277 EM attending physicians, of whom 95 (34%) identified as women, there were notable gender differences in the content of the narrative comments from faculty to residents. Men faculty, compared with women faculty, were more likely to provide either nonspecific comments (115 of 182 [63.2%] vs 40 of 95 [42.1%]), or no comments (3387 of 10 496 [32.3%] vs 1169 of 4548 [25.7%]; P < .001) to men and women residents. Compared with men residents, more women residents were told that they were performing below level by men and women faculty (36 of 113 [31.9%] vs 43 of 170 [25.3%]), with the most common theme including lack of confidence with procedural skills.Conclusions and relevanceIn this qualitative study of narrative comments provided by EM attending physicians to residents, multiple modifiable contributors to gender disparities in assessment were identified, including the presence, content, and specificity of comments. Among women residents, procedural competency was associated with being conflated with procedural confidence. These findings can inform interventions to improve parity in assessment across graduate medical education.

Dataset Information

Using Learning Analytics to Examine Differences in Assessment Forms From Continuous Versus Episodic Supervisors of Family Medicine Residents.

Background

Objective

Methods

Results

Conclusions

Publications

Using Learning Analytics to Examine Differences in Assessment Forms From Continuous Versus Episodic Supervisors of Family Medicine Residents.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets