Misunderstanding the Match: Do Students Create Rank Lists Based on True Preferences?
ABSTRACT: INTRODUCTION:The "stable marriage" algorithm underlying the National Residency Match Program (NRMP) has been shown to create optimal outcomes when students submit true preference lists. Previous research has shown students may allow external information to affect their rank lists. The objective of this study was to determine whether medical students consistently make rank lists that reflect their true preferences. METHODS:A voluntary online survey was sent to third-year students at a single midwestern medical school. Students were given hypothetical scenarios that either should or should not affect their true residency preferences and rated the importance of six factors to their final rank list. The survey was edited by a group of education scholars and revised based on feedback from a pilot with current postgraduate year 1 residents. RESULTS:Of 175 students surveyed, 140 (80%) responded; 63% (88/140) reported that their "perceived competitiveness" would influence their rank list at least a "moderate amount. Of 135 students, 31 (23%) moved a program lower on their list if they learned they were ranked "low" by that program, while 6% (8/135) of respondents moved a program higher if they learned they were ranked "at the top of the list." Participants responded similarly (? = 0.71) when presented with scenarios asking what they would do vs what a classmate should do. CONCLUSION:Students' hypothetical rank lists did not consistently match their true residency preferences. These results may stem from a misunderstanding of the Match algorithm. Medical schools should consider augmenting explicit education related to the NRMP Match algorithm to ensure optimal outcomes for students.
Project description:INTRODUCTION:The National Residency Matching Program (NRMP) allows post-interview contact between residency applicants and residency programs. Thank-you communications represent one of the most common forms, but data on their value to applicants and program directors (PD) are limited. The objective of this study was to assess the effect of thank-you communications on applicant- and residency-program rank lists. METHODS:Two anonymous, voluntary surveys were sent after the 2018 NRMP Match, one to applicants who were offered an interview at a single academic site in the 2017-2018 Match cycle, and one to EM PDs nationwide. The surveys were designed in conjunction with a nationally-recognized survey center and piloted and revised based on feedback from residents and faculty. RESULTS:Of 196 residency applicants, 97 (49.5%) responded to the survey. Of these, 73/95 (76.8%) reported sending thank-you communications. Twenty-two of 73 (30%) stated that they sent thank-you communications to improve their spot on a program's rank list; and 16 of 73 (21.9%) reported that they changed their rank list based upon the responses they received to their thank-you communications. Of 163 PDs, 99 (60.7%) responded to the survey. Of those PDs surveyed, 22.6% reported that an applicant could be moved up their program's rank list and 10.8% reported that an applicant could move down a program's rank list based on their thank-you communications (or lack thereof). CONCLUSION:The majority of applicants to EM are sending thank-you communications. A significant minority of applicants and PDs changed their rank list due to post-interview thank-you communications.
Project description:Introduction:Each year residency programs expend considerable effort ranking applicants for the National Residency Matching Program (NRMP). We explored the relationship between residents' NRMP rank list position as generated at our institution and their performance in residency and post-graduation to determine whether such efforts are justified. Methods:Faculty who were present for the 10 consecutive study years at an allopathic emergency medicine residency retrospectively evaluated residents on their overall performance, medical knowledge, and interpersonal skills. Residency graduates were surveyed regarding their current position, hours of clinical practice, academic, teaching and leadership roles, and publications. We compared match position to performance using graphical techniques as the primary form of analysis. Results:Ten faculty evaluated the 107 residents who graduated from the program during these 10 years by class year. Eighty-four residents responded to the survey. In general, we found little correlation between NRMP rank and faculty rank of resident performance. There was also little correlation between position in the NRMP rank list and the probability of having an academic career, publishing research, or having a teaching or leadership role. Conclusion:We found that the position on our NRMP rank list was of little value in predicting which residents would do best in residency or take on academic or leadership roles once graduated. Residencies should evaluate the processes they use to generate their rank list to determine whether the ranking process is sufficiently predictive to warrant the effort expended.
Project description:Background :Interview experiences and postinterview communication during the residency match process can cause distress for applicants, and deserve further study. Objective :We both quantified and qualified the nature of various interview behaviors during the 2015-2016 National Resident Matching Program (NRMP) Match and collected applicant perspectives on postinterview communication and preferences for policy change. Methods :An anonymous, 31-question survey was sent to residency candidates applying to 8 residency programs at a single academic institution regarding their experiences at all programs where they interviewed. Results :Of 6693 candidates surveyed, 2079 (31%) responded. Regarding interview experiences, applicants reported being asked at least once about other interviews, marital status, and children at the following rates: 72%, 38%, and 17%, respectively, and such questions arose at a reported mean of 25%, 14%, and 5% of programs, respectively. Female applicants were more frequently asked about children than male applicants (22% versus 14%, P < .0001). Overall, 91% of respondents engaged in postinterview communication. A total of 70% of respondents informed their top program that they had ranked it highly; 70% of this subset reported associated distress, and 78% reported doing this to improve match success. A total of 71% would feel relief if postinterview communication was actively discouraged, and 51% would prefer applicants to be prohibited from notifying programs of their rank. Conclusions :Applicants to several residency programs reported being asked questions that violate the NRMP Code of Conduct. The majority of applicants would prefer postinterview communication to be more regulated and less prevalent.
Project description:INTRODUCTION: Residency programs seek to match the best candidates with their positions. To avoid ethical conflicts in this process, the National Residency Matching Program (NRMP or Match) has rules regarding appropriate conduct, including guidelines on contact between candidates and programs. Our study examined communication between obstetrics and gynecology (Ob-Gyn) programs and residency candidates after interviewing and prior to ranking. METHODS: Ob-Gyn program directors in the United States were sent a self-administered survey via e-mail. Data were collected and analyzed using descriptive methods to examine communication practices of these programs. RESULTS: The response rate was 40%. The findings showed that respondents had variable interpretations of the NRMP rules and suggest that programs may be communicating their match intentions especially to favored candidates. Respondents' open text comments highlighted program directors' frustrations with current NRMP rules. DISCUSSION: NRMP communication rules are intended to minimize pressure on residency candidates. Our findings suggest they may be leading to unforeseen stresses on program directors and candidates. CONCLUSIONS: As educational leaders in medicine, we must consider what professional communications are acceptable without increasing the pressure on candidates during the ranking and match process.
Project description:Board scores are an important aspect of an emergency medicine (EM) residency application. Residency directors use these standardized tests to objectively evaluate an applicant's potential and help decide whether to interview a candidate. While allopathic (MD) students take the United States Medical Licensing Examination (USMLE), osteopathic (DO) students take the Comprehensive Osteopathic Medical Licensing Examination (COMLEX). It is difficult to compare these scores. Previous literature proposed an equation to predict USMLE based on COMLEX. Recent analyses suggested this may no longer be accurate. DO students applying to allopathic programs frequently ask whether they should take USMLE to overcome this potential disadvantage. The objective of the study is to compare the likelihood to match of DO applicants who reported USMLE to those who did not, and to clarify how important program directors consider it is whether or not an osteopathic applicant reported a USMLE score.We conducted a review of Electronic Residency Application Service (ERAS) and National Resident Matching Program (NRMP) data for 2010-2011 in conjunction with a survey of EM residency programs. We reviewed the number of allopathic and osteopathic applicants, the number of osteopathic applicants who reported a USMLE score, and the percentage of successful match. We compared the percentage of osteopathic applicants who reported a USMLE score who matched compared to those who did not report USMLE. We also surveyed allopathic EM residency programs to understand how important it is that osteopathic (DO) students take USMLE.There were 1,482 MD students ranked EM programs; 1,277 (86%, 95% CI 84.3-87.9) matched. There were 350 DO students ranked EM programs; 181 (52%, 95% CI 46.4-57.0) matched (difference=34%, 95% CI 29.8-39.0, p<0.0001). There were 208 DO students reported USMLE; 126 (61%, 95% CI 53.6-67.2) matched. 142 did not report USMLE; 55 (39%, 95% CI 30.7-47.3) matched (difference=22%, 95% CI 11.2-32.5, p<0.0001). Survey results: 39% of program directors reported that it is extremely important that osteopathic students take USMLE, 38% stated it is somewhat important, and 22% responded not at all important.DO students who reported USMLE were more likely to match. DO students applying to allopathic EM programs should consider taking USMLE to improve their chances of a successful match.
Project description:BACKGROUND: Medical students rank residency programs as part of the selection process in the National Resident Matching Program, also known as the match. Applicants to medical residency positions are protected against discriminatory employment practices by federal employment laws. OBJECTIVES: To explore students' recall of being asked potentially illegal or discriminatory questions during the selection interview, and whether these questions affected students' ranking of the programs in the match. METHODS: Fourth-year medical students from a single medical school were surveyed after the match. Students were questioned about their recall of the frequency of potentially illegal or discriminatory interview questions and their effect on the program's rank. RESULTS: Ninety percent of the 63 respondents in the study remember being asked at least one potentially discriminatory question. Among these, students were asked about their marital status (86%), about children (31%), about plans for pregnancy (10%), where they were born (54%) and/or about their national origin (15%), and about religious and ethical beliefs (24%). The majority of students did not think the questions changed their decision to rank the program, although the questions changed the way some students ranked the program, either lowering or raising the rank. CONCLUSION: Nearly all students reported that they were asked at least one potentially discriminatory question, although these questions for the most part do not appear to affect whether they ranked the programs.
Project description:Introduction:The Residency Match is becoming more competitive each year, and more than ever, medical students must prove themselves in the residency interview. Data from the 2018 National Residency Matching Program's Program Director Survey highlight the importance of the interview on Match performance. We developed a residency interview training program with the goal of preparing medical students for residency interviews, and we assessed the impact of the training on medical student confidence. Methods:Our residency interview training program includes (1) a short didactic session on the residency interview process, (2) an informational packet with commonly asked questions, (3) two 20-minute practice (mock) interviews, (4) 10 minutes of face-to-face feedback from interviewers, (5) a facilitated group debriefing, and (6) access to a participant's two videotaped interviews and a guided self-assessment. To evaluate the effectiveness of our program, we assessed student confidence with a pre- and postsurvey. Results:We have been running our residency interview training program since 2014. Over the last 5 years, 73 fourth-year medical students have participated. When polled after completing their first authentic residency interview, students felt they had more knowledge of the interview process, better preparation, and more confidence in their skills to interview successfully; they also believed that the program improved their interview performance. Discussion:Performance during the residency interview is the most important factor used by residency programs to rank applicants. Using our residency interview training program, medical students can improve their confidence prior to interviews.
Project description:Background:Residency program prestige is an important variable medical students consider when creating their rank list. Doximity Residency Navigator is a ranking system that previous reports have shown significantly influences medical student application decisions. Doximity's use of peer nomination as a central component of its methodology for determining program rank has drawn criticism for its lack of objectivity. Doximity has not published information regarding how peer nomination and more objective measures are statistically weighted in reputation calculation. Objective:This study assesses whether a strong negative correlation exists between residency program size and Doximity ranking. Methods:A cross-sectional study of Doximity residency rankings from the 2018-2019 academic year was conducted. Data extracted from Doximity included program rank, size, and age. Data were additionally collected from the Blue Ridge Institute for Medical Research, National Institutes of Health, funding in 2018 and the US News & World Report Best Medical Schools 2019-2020. A multivariable linear regression model was used that included Doximity ranking as the outcome variable and residency program size as the predictor variable with adjustment for the aforementioned variables. Results:Sixteen of the 28 specialties on Doximity were included in the analysis, representing 3388 unique residency programs. After adjustment for covariates, residency program size was a significant predictor of Doximity ranking (? = -1.84; 95% CI -2.01 to -1.66, P < .001). Conclusions:These findings support the critique that the Doximity reputation ranking system may favor larger residency programs. More transparency for Doximity reputation ranking algorithm is warranted.
Project description:BACKGROUND: Residency interviews can place significant time and financial burdens on applicants. OBJECTIVE: To determine whether the use of Skype as a screening tool during interview season in a family medicine residency is cost-effective and time-efficient for the applicant and the residency program. METHODS: We surveyed 2 groups of medical students during interviews for our family medicine program. Thirty-two students were interviewed via our face-to-face, traditional interview (TI) process, and 10 students, the second group, who did not meet the program's standard interview selection criteria for TI, underwent our Skype interview (SI) process. RESULTS: Using an unpaired t test, we found that the applicants' costs of an SI were significantly less than a TI, $566 (95% confidence interval [CI] $784-$349, P < .001). Direct cash savings plus indirect salary savings to the program were $5,864, with a time savings of 7 interview days. Three of the applicants who were participants in the SI limb of the study were in our final rank order list. CONCLUSIONS: For interviewing in family medicine residencies, use of Skype may be a cost-effective and time-efficient screening tool for both the applicant and the program. Alternate uses of SI may include the time-sensitive, postmatch Supplemental Offer and Acceptance Program.
Project description:BACKGROUND: Many medical students find choosing a residency challenging. There are several steps involved, including determining one's own priorities, arranging electives, choosing a training program and site, and preparing an in-depth application and a rank order list. Many resources are available to assist students, including the Canadian Resident Matching Service website, program websites, career counselors, career information sessions, mentors, peers, family/friends, and the Canadian Medical Residency Guide. Our study explored the relative impact of these resources on the career decision-making process. METHOD: We invited medical students in their final year at 12 Canadian medical schools to complete an online survey. Questions included identifying the relative utility of resources in the context of each component of the decision-making process. Responses were analyzed using descriptive statistics. RESULTS: The response rate was 71% (1076 of 1518). Overall, mentors, family/friends, and peers had the most impact on students' decision making. Career counselors, websites, and the Canadian Medical Residency Guide had much less impact. Family/friends were most frequently cited as essential to the process; however, family/friends and peers were equal in having some impact. CONCLUSIONS: Our findings suggest that students are most influenced by family, friends, and peers, who are not involved in the formal residency selection effort. Appreciating the impact of these influences on students' decision making is important to understanding how they decide on their future careers. The study supports continuation of mentorship programs. Future work could focus on qualitative research to further characterize resource use.