Dataset Information

Analysis of narrative assessments of internal medicine resident performance: are there differences associated with gender or race and ethnicity?

ABSTRACT:

Background

Equitable assessment is critical in competency-based medical education. This study explores differences in key characteristics of qualitative assessments (i.e., narrative comments or assessment feedback) of internal medicine postgraduate resident performance associated with gender and race and ethnicity.

Methods

Analysis of narrative comments included in faculty assessments of resident performance from six internal medicine residency programs was conducted. Content analysis was used to assess two key characteristics of comments- valence (overall positive or negative orientation) and specificity (detailed nature and actionability of comment) - via a blinded, multi-analyst approach. Differences in comment valence and specificity with gender and race and ethnicity were assessed using multilevel regression, controlling for multiple covariates including quantitative competency ratings.

Results

Data included 3,383 evaluations with narrative comments by 597 faculty of 698 residents, including 45% of comments about women residents and 13.2% about residents who identified with race and ethnicities underrepresented in medicine. Most comments were moderately specific and positive. Comments about women residents were more positive (estimate 0.06, p 0.045) but less specific (estimate - 0.07, p 0.002) compared to men. Women residents were more likely to receive non-specific, weakly specific or no comments (adjusted OR 1.29, p 0.012) and less likely to receive highly specific comments (adjusted OR 0.71, p 0.003) or comments with specific examples of things done well or areas for growth (adjusted OR 0.74, p 0.003) than men. Gendered differences in comment specificity and valence were most notable early in training. Comment specificity and valence did not differ with resident race and ethnicity (specificity: estimate 0.03, p 0.32; valence: estimate - 0.05, p 0.26) or faculty gender (specificity: estimate 0.06, p 0.15; valence: estimate 0.02 p 0.54).

Conclusion

There were significant differences in the specificity and valence of qualitative assessments associated with resident gender with women receiving more praising but less specific and actionable comments. This suggests a lost opportunity for well-rounded assessment feedback to the disadvantage of women.

SUBMITTER: Klein R

PROVIDER: S-EPMC10795394 | biostudies-literature | 2024 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Analysis of narrative assessments of internal medicine resident performance: are there differences associated with gender or race and ethnicity?

Klein Robin R Snyder Erin D ED Koch Jennifer J Volerman Anna A Alba-Nguyen Sarah S Julian Katherine A KA Thompson Vanessa V Ufere Nneka N NN Burnett-Bowie Sherri-Ann M SM Kumar Anshul A White Bobbie Ann A BAA Park Yoon Soo YS Palamara Kerri K

BMC medical education 20240117 1

<h4>Background</h4>Equitable assessment is critical in competency-based medical education. This study explores differences in key characteristics of qualitative assessments (i.e., narrative comments or assessment feedback) of internal medicine postgraduate resident performance associated with gender and race and ethnicity.<h4>Methods</h4>Analysis of narrative comments included in faculty assessments of resident performance from six internal medicine residency programs was conducted. Content anal ...[more]

PMID: 38233807

Similar Datasets

Project description:ImportancePrevious studies have demonstrated sex-specific disparities in performance assessments among emergency medicine (EM) residents. However, less work has focused on intersectional disparities by ethnoracial identity and sex in resident performance assessments.ObjectiveTo estimate intersectional sex-specific ethnoracial disparities in standardized EM resident assessments.Design, setting, and participantsThis retrospective cohort study used data from the Association of American Medical Colleges and the Accreditation Council for Graduate Medical Education Milestones (Milestones) assessments to evaluate ratings for EM residents at 128 EM training programs in the US. Statistical analyses were conducted in June 2020 to January 2023.ExposureTraining and assessment environments in EM residency programs across comparison groups defined by ethnoracial identity (Asian, White, or groups underrepresented in medicine [URM], ie, African American/Black, American Indian/Alaska Native, Hispanic/Latine, and Native Hawaiian/Other Pacific Islander) and sex (female/male).Main outcomes and measuresMean Milestone scores (scale, 0-9) across 6 core competency domains: interpersonal and communications skills, medical knowledge, patient care, practice-based learning and improvement, professionalism, and system-based practice. Overall assessment scores were calculated as the mean of the 6 competency scores.ResultsThe study sample comprised 128 ACGME-accredited programs and 16 634 assessments for 2708 EM residents of which 1913 (70.6%) were in 3-year and 795 (29.4%) in 4-year programs. Most of the residents were White (n = 2012; 74.3%), followed by Asian (n = 477; 17.6%), Hispanic or Latine (n = 213; 7.9%), African American or Black (n = 160; 5.9%), American Indian or Alaska Native (n = 24; 0.9%), and Native Hawaiian or Other Pacific Islander (n = 4; 0.1%). Approximately 14.3% (n = 386) and 34.6% (n = 936) were of URM groups and female, respectively. Compared with White male residents, URM female residents in 3-year programs were rated increasingly lower in the medical knowledge (URM female score, -0.47; 95% CI, -0.77 to -0.17), patient care (-0.18; 95% CI, -0.35 to -0.01), and practice-based learning and improvement (-0.37; 95% CI, -0.65 to -0.09) domains by postgraduate year 3 year-end assessment; URM female residents in 4-year programs were also rated lower in all 6 competencies over the assessment period.Conclusions and relevanceThis retrospective cohort study found that URM female residents were consistently rated lower than White male residents on Milestone assessments, findings that may reflect intersectional discrimination in physician competency evaluation. Eliminating sex-specific ethnoracial disparities in resident assessments may contribute to equitable health care by removing barriers to retention and promotion of underrepresented and minoritized trainees and facilitating diversity and representation among the emergency physician workforce.

Project description:ImportancePrevious studies have demonstrated racial and ethnic inequities in medical student assessments, awards, and faculty promotions at academic medical centers. Few data exist about similar racial and ethnic disparities at the level of graduate medical education.ObjectiveTo examine the association between race and ethnicity and performance assessments among a national cohort of internal medicine residents.Design, setting, and participantsThis retrospective cohort study evaluated assessments of performance for 9026 internal medicine residents from the graduating classes of 2016 and 2017 at Accreditation Council of Graduate Medical Education (ACGME)-accredited internal medicine residency programs in the US. Analyses were conducted between July 1, 2020, and June 31, 2022.Main outcomes and measuresThe primary outcome was midyear and year-end total ACGME Milestone scores for underrepresented in medicine (URiM [Hispanic only; non-Hispanic American Indian, Alaska Native, or Native Hawaiian/Pacific Islander only; or non-Hispanic Black/African American]) and Asian residents compared with White residents as determined by their Clinical Competency Committees and residency program directors. Differences in scores between Asian and URiM residents compared with White residents were also compared for each of the 6 competency domains as supportive outcomes.ResultsThe study cohort included 9026 residents from 305 internal medicine residency programs. Of these residents, 3994 (44.2%) were female, 3258 (36.1%) were Asian, 1216 (13.5%) were URiM, and 4552 (50.4%) were White. In the fully adjusted model, no difference was found in the initial midyear total Milestone scores between URiM and White residents, but there was a difference between Asian and White residents, which favored White residents (mean [SD] difference in scores for Asian residents: -1.27 [0.38]; P < .001). In the second year of training, White residents received increasingly higher scores relative to URiM and Asian residents. These racial disparities peaked in postgraduate year (PGY) 2 (mean [SD] difference in scores for URiM residents, -2.54 [0.38]; P < .001; mean [SD] difference in scores for Asian residents, -1.9 [0.27]; P < .001). By the final year 3 assessment, the gap between White and Asian and URiM residents' scores narrowed, and no racial or ethnic differences were found. Trends in racial and ethnic differences among the 6 competency domains mirrored total Milestone scores, with differences peaking in PGY2 and then decreasing in PGY3 such that parity in assessment was reached in all competency domains by the end of training.Conclusions and relevanceIn this cohort study, URiM and Asian internal medicine residents received lower ratings on performance assessments than their White peers during the first and second years of training, which may reflect racial bias in assessment. This disparity in assessment may limit opportunities for physicians from minoritized racial and ethnic groups and hinder physician workforce diversity.

Project description:ImportanceIncreasing diversity in the physician workforce is a fulcrum for reducing health disparities. Efforts to increase the diversity in the internal medicine (IM) workforce may improve health equity among an increasingly diverse population with increasing prevalence of chronic disease.ObjectivesTo assess diversity trends in the academic IM workforce and evaluate how well these trends reflected medical student diversity and the changing demographic composition of the general population.Design, setting, and participantsThis secondary analysis of a cross-sectional study analyzed data from January 1, 1980, to December 31, 2018, from the Association of American Medical Colleges Faculty Roster and Applicant Matriculant File, which capture full-time US medical school faculty and matriculants, respectively, and population data through 2017 from the US Census Bureau.Main outcomes and measuresThe study calculated the proportions of women and individuals from racial/ethnic groups that are traditionally underrepresented in medicine (URM) among IM faculty and faculty in all other clinical departments. These data were compared with the proportions of female and URM matriculants in US medical schools and the proportions of women and individuals from underrepresented racial/ethnic groups in the population. The analysis was stratified by sex, race/ethnicity, and intersections of sex and race/ethnicity.ResultsFrom 1980 to 2018, the absolute number of full-time IM faculty increased from 10 964 to 42 547. Although IM was the department classification with the most women faculty, in 2018 it continued to have a lower proportion of women (n = 17 165 [40.3%]) compared with all other clinical departments (n = 48 936 [43.2%]). Among IM faculty, the percentage of URM faculty members more than doubled during the study period (from 4.1% to 9.7%) but still made up only a small portion of faculty members. The percentage of female matriculants among medical school matriculants increased steadily (from 28.7% in 1980 to 51.6% in 2018) and was nearly identical to their population representation in 2017 (50.7% compared with 50.8%). Although the percentage of URM matriculants had nearly doubled since 1980 (from 11.3% to 18.1%), it still lagged far behind the proportion of individuals in the US population who are members of underrepresented racial/ethnic groups (18.1% vs 31.5% in 2017).Conclusions and relevanceThis cross-sectional study found that progress has been made in diversifying academic IM faculty; however, it does not yet reflect the diversity of medical students or the US population. Continued efforts to increase the diversity of the academic IM workforce are needed.

Dataset Information

Analysis of narrative assessments of internal medicine resident performance: are there differences associated with gender or race and ethnicity?

Background

Methods

Results

Conclusion

Publications

Analysis of narrative assessments of internal medicine resident performance: are there differences associated with gender or race and ethnicity?

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets