Dataset Information

A Redesign of the ACGME Resident/Fellow Survey Through Expert Reviews and Cognitive Interviews.

ABSTRACT:

Background

Since 2003, the Accreditation Council for Graduate Medical Education (ACGME) has surveyed residents and fellows in its accredited programs. The Resident/Fellow Survey is a screening and compliance tool that programs can also use for continuous quality improvement. However, stakeholders have reported potential problems with the survey's overall quality and credibility.

Objective

To redesign the 2006 Resident/Fellow Survey using expert reviews and cognitive interviews.

Methods

In 2018-2019, the ACGME redesigned the Resident/Fellow Survey using an iterative validation process: expert reviews (evidence based on content) and cognitive interviews (evidence based on response processes). Expert reviews were conducted by a survey design firm and an ACGME Task Force; cognitive interviews were conducted with a diverse set of 27 residents and fellows.

Results

Expert reviews resulted in 21 new survey items to address the ACGME's updated accreditation requirements; these reviews also led to improvements that align the survey items with evidence-informed standards. After these changes were made, cognitive interviews resulted in additional revisions to sensitive items, item order, and response option order, labels, and conceptual spacing. In all, cognitive interviews led to 11 item deletions and other improvements designed to reduce response error.

Conclusions

Expert reviews and cognitive interviews were used to redesign the Resident/Fellow Survey. The content of the redesigned survey aligns with the updated accreditation requirements and items are written in accordance with evidence-informed standards. Additionally, cognitive interviews resulted in revisions to the survey that seem to improve respondents' comprehension and willingness to respond to individual survey items.

SUBMITTER: Artino AR

PROVIDER: S-EPMC9934830 | biostudies-literature | 2023 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A Redesign of the ACGME Resident/Fellow Survey Through Expert Reviews and Cognitive Interviews.

Artino Anthony R AR Malloy Kathy K Miller Rebecca S RS Kirk Lynne M LM Brigham Timothy P TP

Journal of graduate medical education 20230201 1

<h4>Background</h4>Since 2003, the Accreditation Council for Graduate Medical Education (ACGME) has surveyed residents and fellows in its accredited programs. The Resident/Fellow Survey is a screening and compliance tool that programs can also use for continuous quality improvement. However, stakeholders have reported potential problems with the survey's overall quality and credibility.<h4>Objective</h4>To redesign the 2006 Resident/Fellow Survey using expert reviews and cognitive interviews.<h4 ...[more]

PMID: 36817519

Similar Datasets

Project description:ContextThe ACGME-released revisions to the 2003 duty hour standards.ObjectiveTo review the impact of the 2003 duty hour reform as it pertains to resident and patient outcomes.Data sourcesMedline (1989-May 2010), Embase (1989-June 2010), bibliographies, pertinent reviews, and meeting abstracts.Study selectionWe included studies examining the relationship between the pre- and post-2003 time periods and patient outcomes (mortality, complications, errors), resident education (standardized test scores, clinical experience), and well-being (as measured by the Maslach Burnout Inventory). We excluded non-US studies.Data extractionOne rater used structured data collection forms to abstract data on study design, quality, and outcomes. We synthesized the literature qualitatively and included a meta-analysis of patient mortality.ResultsOf 5,345 studies identified, 60 met eligibility criteria. Twenty-eight studies included an objective outcome related to patients; 10 assessed standardized resident examination scores; 26 assessed resident operative experience. Eight assessed resident burnout. Meta-analysis of the mortality studies revealed a significant improvement in mortality in the post-2003 time period with a pooled odds ratio (OR) of 0.9 (95% CI: 0.84, 0.95). These results were significant for medical (OR 0.91; 95% CI: 0.85, 0.98) and surgical patients (OR 0.86; 95% CI: 0.75, 0.97). However, significant heterogeneity was present (I(2) 83%). Patient complications were more nuanced. Some increased in frequency; others decreased. Outcomes for resident operative experience and standardized knowledge tests varied substantially across studies. Resident well-being improved in most studies.LimitationsMost studies were observational. Not all studies of mortality provided enough information to be included in the meta-analysis. We used unadjusted odds ratios in the meta-analysis; statistical heterogeneity was substantial. Publication bias is possible.ConclusionsSince 2003, patient mortality appears to have improved, although this could be due to secular trends. Resident well-being appears improved. Change in resident educational experience is less clear.

Project description:ImportancePhysicians, especially surgeons, delay family building, and face age-related complications of pregnancy.ObjectiveTo explore factors that are associated with family building decisions among residents and fellows and to understand their thoughts on accessing family building supports, workplace culture, and attitudes toward family planning.Design, setting, and participantsThis qualitative study was conducted from May to August 2022 at postgraduate training sites across Ontario, Canada. Participants included 29 postgraduate learners. Purposive sampling from 80 eligible trainees was used to ensure diverse demographics, including sex, specialty, and institution. Semi-structured individual interviews lasting 30 to 90 minutes were conducted over a virtual platform. Participants were asked to share their thoughts and experiences on family planning in a medical career, family building goals, the role of mentorship, and knowledge of current family planning supports.Main outcomes and measuresThemes associated with residents' perception of family planning within the context of a medical career.ResultsTwenty-nine trainees, ages 25 to 37 years old (22 [75.9%] were female, 26 [89.7%] were heterosexual; median [range] age, 30 [25-37] years) from various institutions took part in this study. The participants included both 24 residents (82.8%) and 5 fellows (17.2%) and 8 (27.6%) were from surgical specialties, 3 (10.3%) from pediatrics and subspecialties, 4 (13.8%) from internal medicine and subspecialties, 2 (6.9%) from obstetrics and gynecology, 3 (10.3%) family medicine, 2 (6.9%) from anesthesia, and 7 (24.1%) other medical specialties. Four themes were identified: (1) tension between role as a physician and role as a parent; (2) impact of role models and mentorship on family planning choices; (3) family building is discouraged during training, especially in surgical specialties; and (4) need for tangible family planning supports in training.Conclusions and relevanceIn this qualitative study of residents' and fellows' perspectives on family building, trainees did not perceive residency and fellowship programs as supportive for individuals who want to family build during training leading to personal and professional dissonance. Trainees identified that more tangible supports, transparent information, and mentorship may improve a trainees' feeling of support.

Dataset Information

A Redesign of the ACGME Resident/Fellow Survey Through Expert Reviews and Cognitive Interviews.

Background

Objective

Methods

Results

Conclusions

Publications

A Redesign of the ACGME Resident/Fellow Survey Through Expert Reviews and Cognitive Interviews.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets