Project description:BackgroundMedical specialty certification exams are high-stakes summative assessments used to determine which doctors have the necessary skills, knowledge, and attitudes to treat patients independently. Such exams are crucial for patient safety, candidates' career progression and accountability to the public, yet vary significantly among medical specialties and countries. It is therefore of paramount importance that the quality of specialty certification exams is studied in the scientific literature.MethodsIn this systematic literature review we used the PICOS framework and searched for papers concerning medical specialty certification exams published in English between 2000 and 2020 in seven databases using a diverse set of search term variations. Papers were screened by two researchers independently and scored regarding their methodological quality and relevance to this review. Finally, they were categorized by country, medical specialty and the following seven Ottawa Criteria of good assessment: validity, reliability, equivalence, feasibility, acceptability, catalytic and educational effect.ResultsAfter removal of duplicates, 2852 papers were screened for inclusion, of which 66 met all relevant criteria. Over 43 different exams and more than 28 different specialties from 18 jurisdictions were studied. Around 77% of all eligible papers were based in English-speaking countries, with 55% of publications centered on just the UK and USA. General Practice was the most frequently studied specialty among certification exams with the UK General Practice exam having been particularly broadly analyzed. Papers received an average of 4.2/6 points on the quality score. Eligible studies analyzed 2.1/7 Ottawa Criteria on average, with the most frequently studied criteria being reliability, validity, and acceptability.ConclusionsThe present systematic review shows a growing number of studies analyzing medical specialty certification exams over time, encompassing a wider range of medical specialties, countries, and Ottawa Criteria. Due to their reliance on multiple assessment methods and data-points, aspects of programmatic assessment suggest a promising way forward in the development of medical specialty certification exams which fulfill all seven Ottawa Criteria. Further research is needed to confirm these results, particularly analyses of examinations held outside the Anglosphere as well as studies analyzing entire certification exams or comparing multiple examination methods.
Project description:ImportanceIdentification of patients with chronic kidney disease (CKD) with high risk of progression to kidney failure can help ensure they receive appropriate and effective nephrology care.ObjectiveTo examine whether patients with CKD at various levels of kidney failure risk receive nephrology care within 1 year of established risk.Design, setting, and participantsThis population-based, retrospective cohort study collected nationwide administrative health claims data from 156 733 adult patients who met the Kidney Disease: Improving Global Outcomes initiative CKD diagnostic criteria between January 1, 2012, and December 31, 2019, and had an available urine albumin to creatinine ratio within 90 days of a serum creatinine laboratory test. Patients with a history of dialysis or kidney transplant, a prior visit with a nephrologist in the past year, or palliative care billing codes or those who died or disenrolled within 1 year of the albumin to creatinine ratio measurement were excluded. Data analysis was performed from September 10, 2022, to February 14, 2022.ExposuresKidney failure risk computed with the 5-year Kidney Failure Risk Equation.Main outcomes and measuresThe main outcome was nephrology care rates across tiers of kidney failure risk, estimated as the proportion of individuals having a nephrologist visit within 1 year after index time.ResultsThe study population consisted of 156 733 patients with CKD (mean [SD] age, 74.6 [8.4] years; 91 906 [58.6%] female; 86 457 [55.2%] White). A total of 106 004 patients (67.6%) had a low (≤1%) 5-year risk of kidney failure. Nephrology visit rates increased with higher kidney failure risk. Among the 137 highest-risk patients, 79 (57.7%; 95% CI, 48.4%-64.7%) had a nephrology visit. Among 7730 patients with risk above a 10% threshold, 3208 (41.5%; 95% CI, 40.3%-42.4%) had a nephrology visit.Conclusions and relevanceThis study's findings suggest that nearly half of patients with CKD at high risk of progressing to kidney failure do not have a nephrologist visit within 1 year of established risk. These findings have implications in the design of risk-based guidelines for referral and in the practice of delivering nephrology care to patients with CKD.
Project description:BackgroundAccurate ICD-10 coding is crucial for healthcare reimbursement, patient care, and research. AI implementation, like ChatGPT, could improve coding accuracy and reduce physician burden. This study assessed ChatGPT's performance in identifying ICD-10 codes for nephrology conditions through case scenarios for pre-visit testing.MethodsTwo nephrologists created 100 simulated nephrology cases. ChatGPT versions 3.5 and 4.0 were evaluated by comparing AI-generated ICD-10 codes against predetermined correct codes. Assessments were conducted in two rounds, 2 weeks apart, in April 2024.ResultsIn the first round, the accuracy of ChatGPT for assigning correct diagnosis codes was 91 and 99% for version 3.5 and 4.0, respectively. In the second round, the accuracy of ChatGPT for assigning the correct diagnosis code was 87% for version 3.5 and 99% for version 4.0. ChatGPT 4.0 had higher accuracy than ChatGPT 3.5 (p = 0.02 and 0.002 for the first and second round respectively). The accuracy did not significantly differ between the two rounds (p > 0.05).ConclusionChatGPT 4.0 can significantly improve ICD-10 coding accuracy in nephrology through case scenarios for pre-visit testing, potentially reducing healthcare professionals' workload. However, the small error percentage underscores the need for ongoing review and improvement of AI systems to ensure accurate reimbursement, optimal patient care, and reliable research data.
Project description:ObjectiveTo analyze the accuracy of ChatGPT-generated responses to common rhinologic patient questions.MethodsTen common questions from rhinology patients were compiled by a panel of 4 rhinology fellowship-trained surgeons based on clinical patient experience. This panel (Panel 1) developed consensus "expert" responses to each question. Questions were individually posed to ChatGPT (version 3.5) and its responses recorded. ChatGPT-generated responses were individually graded by Panel 1 on a scale of 0 (incorrect) to 3 (correct and exceeding the quality of expert responses). A 2nd panel was given the consensus and ChatGPT responses to each question and asked to guess which response corresponded to which source. They then graded ChatGPT responses using the same criteria as Panel 1. Question-specific and overall mean grades for ChatGPT responses, as well as interclass correlation coefficient (ICC) as a measure of interrater reliability, were calculated.ResultsThe overall mean grade for ChatGPT responses was 1.65/3. For 2 out of 10 questions, ChatGPT responses were equal to or better than expert responses. However, for 8 out of 10 questions, ChatGPT provided responses that were incorrect, false, or incomplete based on mean rater grades. Overall ICC was 0.526, indicating moderate reliability among raters of ChatGPT responses. Reviewers were able to discern ChatGPT from human responses with 97.5% accuracy.ConclusionThis preliminary study demonstrates overall near-complete and variably accurate responses provided by ChatGPT to common rhinologic questions, demonstrating important limitations in nuanced subspecialty fields.
Project description:ObjectivesThe specialty of emergency medicine (EM) is experiencing a significant decrease in student interest. In addition, women are historically underrepresented within the specialty at all levels of training and practice. We sought to understand how clinical experiences and perceptions of EM influence specialty selection by medical students, particularly women.MethodsUsing a constructivist grounded theory approach, we analyzed semistructured interviews with senior medical students who considered EM as a specialty. We used purposive sampling to recruit from diverse learning environments and represent a variety of experiences. Participants reflected on their specialty selection process and experiences in EM including their perceived acceptance in the work environment.ResultsTwenty-five medical students from 11 geographically diverse schools participated. A total of 68% (17/25) identified as women. The majority (21/25, 84%) planned on applying to EM residency. We identified four major themes: (1) distressing interpersonal interactions with patients and the ED care team negatively affect students; (2) EM culture includes behaviors that are perceived as exclusionary; (3) beliefs about the attributes of an ideal EM physician and the specialty itself have a gendered nature; and (4) ease of access to mentors, representation, and early exposure to EM environment increased interest in specialty.ConclusionsOur participants express that EM causes challenges for students to accept the norms of behavior in the field, which is an essential element in joining a group and professional identity formation. In addition, we raise concern that gendered perceptions and language may send exclusionary environmental cues that may negatively impact recruitment of a diverse physician workforce.
Project description:BackgroundNarrative medicine is a well-recognized and respected approach to care. It is now found in medical school curricula and widely implemented in practice. However, there has been no analysis of the perception and usage of narrative medicine across different medical specialties and whether there may be unique recommendations for implementation based upon specialty. The aims of this study were to explore these gaps in research.MethodsFifteen senior physicians who specialize in internal medicine, pediatrics, or surgery (5 physicians from each specialty) were interviewed in a semi-structured format about the utilization, benefits, drawbacks (i.e., negative consequences), and roles pertaining to narrative medicine. Qualitative content analysis of each interview was then performed.ResultsThree themes emerged from our analysis: roles, practice, and outcomes. Through these themes we examined the importance, utilization, barriers, benefits, and drawbacks of narrative medicine. There was consensus that narrative medicine is an important tool in primary care. Primary care physicians (general internists and general pediatricians) also believed that narrative medicine is not as important for non-primary care providers. However, non-primary care providers (surgeons) generally believed narrative medicine is valuable in their practice as well. Within specialties, providers' choice of language varied when trying to obtain patients' narratives, but choice in when to practice narrative medicine did not differ greatly. Among specialties, there was more variability regarding when to practice narrative medicine and what barriers were present. Primary care physicians primarily described barriers to eliciting a patient's narrative to involve trust and emotional readiness, while surgeons primarily described factors involving logistics and patient data as barriers to obtaining patients' narratives. There was broad agreement among specialties regarding the benefits and drawbacks of narrative medicine.ConclusionsThis study sheds light on the shared and unique beliefs in different specialties about narrative medicine. It prompts important discussion around topics such as the stereotypes physicians may hold about their peers and concerns about time management. These data provide some possible ideas for crafting narrative medicine education specific to specialties as well as future directions of study.
Project description:BackgroundOver the past decade, nephrology has experienced a 43% decline in the number of fellowship applicants. Previous studies examining why residents choose a fellowship program cite lack of exposure as a main factor having an effect against a career in nephrology; however, no studies have surveyed the undergraduate population to inquire whether they recognize nephrology as a medical specialty compared with other medical specialties. We conducted a survey at a primarily undergraduate institution in the Southeast United States to test whether undergraduate students identified the word "nephrology."MethodsA total of 274 undergraduates responded to a survey that requested them to select every medical specialty that they recognized by name (15 real specialties and one fictitious specialty). Demographics regarding sex, race, collegiate level, high school location, premedical track, and household income were collected. Correlations between survey findings and rates of application and average salary per specialty were assessed.ResultsOut of 15 medical specialties, nephrology (29%) and pulmonology (40%) were the least recognized. Pediatrics (97%) and surgery (97%) ranked highest. Sex, race, collegiate level, and household income were not different between those students who recognized "nephrology" and those who did not. Premedical students were about twice as likely to have recognized nephrology versus nonpremedical students (49% versus 22%, respectively; P<0.001). STEM majors were about twice as likely to identify nephrology versus non-STEM majors (40% versus 20%, respectively; P<0.001). The proportion of undergraduate students who recognized a specific medical specialty significantly correlated only with the number of US applicants per fellowship position across different medical specialties in 2020 (P<0.05).ConclusionsOn the basis of word association alone, nephrology is the one of the least recognized specialties by undergraduates. The discrepancy between nephrology and other specialties highlights a gap in name recognition at an early career stage, even among premedical students.
Project description:BackgroundFewer trainees are choosing to pursue nephrology. Only 60.1% of positions filled in the 2018 fellowship Match, which is concerning given the rising prevalence of end-stage kidney disease. Identifying factors influential in career choices is critical to inform focused approaches to recruit qualified applicants.MethodsTo understand perceptions of nephrology and assess factors influential in specialty choice among early career trainees, an anonymous survey was distributed to upper-level medical students and internal medicine residents at programs identified through the American Association of Medical Colleges (AAMC) and American Medical Association's Fellowship and Residency Electronic Interactive Database (FREIDA).ResultsOf 4199 recipients, 644 (15.3%) participants responded, including 315 upper-level medical students, 308 residents, and three chief residents from 30 institutions. An interest in the subject was the most critical factor in selecting a specialty (92%). Other key factors included a suitable work-life balance (73%), access to mentors (70%), and subject exposure (66%). Lack of interest was the most frequently-cited reason to forgo a nephrology fellowship (79%), followed by concerns regarding remuneration (43%), work-life balance (39%), and subject exposure (32%). In free-text responses, several participants described frustration with managing patients on hemodialysis and desired combined training with specialties such as critical care. Respondents who had considered nephrology at any point cited an interest in physiology or interface with a mentor as key driving factors.ConclusionsA lack of interest in and exposure to the subject, perceptions of poor earning potential and patient nonadherence, and concerns regarding work-life balance were influential in participants' decisions to forgo nephrology training. Incorporating novel educational tools and broadening the scope of the nephrology elective, highlighting ongoing areas of clinical and research innovation, expanding opportunities for interdisciplinary collaboration and procedural skills, and cultivating strategies to reduce burnout may be useful areas on which to focus future recruitment efforts.