Predicting the "strugglers": a case-control study of students at Nottingham University Medical School.
ABSTRACT: OBJECTIVE: To identify potential predictors of undergraduate students who struggle during their medical training. DESIGN: Case-control study. Cases were students who had experienced academic or personal difficulties that affected their progression on the course ("strugglers"). Controls were selected at random from the corresponding year cohorts, using a ratio of four controls for each struggler. SETTING: University of Nottingham Medical School. PARTICIPANTS: Students who entered the course over five consecutive years. MAIN OUTCOME MEASURES: Likelihood ratios for independent risk factors for struggling on the course RESULTS: 10-15% of each year's student intake were identified as strugglers. Significant independent predictors of students being in this category were negative comments in the academic reference (likelihood ratio 2.25, 95% confidence intervals 1.44 to 3.50), lower mean examination grade at A level (2.19, 1.37 to 3.51), and the late offer of a place (1.98, 1.19 to 3.30). Male sex was a less significant risk factor (1.70, 1.09 to 2.65) as was a lower grade at GCSE science (2.13, 1.12 to 4.05). In UK students whose ethnicity was known, not being white was a significant predictor of struggling (2.77, 1.52 to 5.05) but the presence of negative comments was not. Age at entry to the course and the possession of a previous degree were not predictive. CONCLUSIONS: Our results support retention of existing selection practices relating to academic achievement and critical review of students' references. We plan to undertake further investigation of the reasons why some students, including men, those with late offers and those from ethnic minority backgrounds, may do less well on the Nottingham course.
Project description:<h4>Background</h4>The UK Clinical Aptitude Test (UKCAT) was introduced in 2006 as an additional tool for the selection of medical students. It tests mental ability in four distinct domains (Quantitative Reasoning, Verbal Reasoning, Abstract Reasoning, and Decision Analysis), and the results are available to students and admissions panels in advance of the selection process. As yet the predictive validity of the test against course performance is largely unknown.The study objective was to determine whether UKCAT scores predict performance during the first two years of the 5-year undergraduate medical course at Nottingham.<h4>Methods</h4>We studied a single cohort of students, who entered Nottingham Medical School in October 2007 and had taken the UKCAT. We used linear regression analysis to identify independent predictors of marks for different parts of the 2-year preclinical course.<h4>Results</h4>Data were available for 204/260 (78%) of the entry cohort. The UKCAT total score had little predictive value. Quantitative Reasoning was a significant independent predictor of course marks in Theme A ('The Cell'), (p = 0.005), and Verbal Reasoning predicted Theme C ('The Community') (p < 0.001), but otherwise the effects were slight or non-existent.<h4>Conclusion</h4>This limited study from a single entry cohort at one medical school suggests that the predictive value of the UKCAT, particularly the total score, is low. Section scores may predict success in specific types of course assessment.The ultimate test of validity will not be available for some years, when current cohorts of students graduate. However, if this test of mental ability does not predict preclinical performance, it is arguably less likely to predict the outcome in the clinical years. Further research from medical schools with different types of curriculum and assessment is needed, with longitudinal studies throughout the course.
Project description:BACKGROUND: An earlier study at Nottingham suggested that 10-15% of the medical student intake was likely to fail completely or have substantial problems on the course. This is a problem for the students, the Faculty, and society as a whole. If struggling students could be identified early in the course and additional pastoral resources offered, some of this wastage might be avoided. An exploratory case study was conducted to determine whether there were common indicators in the early years, over and above academic failure, that might aid the identification of students potentially at risk. METHODS: The study group was drawn from five successive cohorts. Students who had experienced difficulties were identified in any of four ways: from Minutes of the Academic Progress Committee; by scanning examination lists at key stages (end of the first two years, and finals at the end of the clinical course); from lists of students flagged to the Postgraduate Deanery as in need of extra monitoring or support; and from progress files of those who had left the course prematurely. Relevant data were extracted from each student's course progress file into a customised database. RESULTS: 1188 students were admitted over the five years. 162 (14%) were identified for the study, 75 of whom had failed to complete the course by October 2010. In the 87 who did graduate, a combination of markers in Years 1 and 2 identified over half of those who would subsequently have the most severe problems throughout the course. This 'toolkit' comprised failure of 3 or more examinations per year, an overall average of <50%, health or social difficulties, failure to complete Hepatitis B vaccination on time, and remarks noted about poor attitude or behaviour. CONCLUSIONS: A simple toolkit of academic and non-academic markers could be used routinely to help identify potential strugglers at an early stage, enabling additional support and guidance to be given to these students.
Project description:BACKGROUND: The UK Clinical Aptitude Test (UKCAT) was introduced in 2006 as an additional tool for the selection of medical students. It tests mental ability in four distinct domains (Verbal Reasoning, Quantitative Reasoning, Abstract Reasoning, and Decision Analysis), and the results are available to students and admission panels in advance of the selection process. Our first study showed little evidence of any predictive validity for performance in the first two years of the Nottingham undergraduate course.The study objective was to determine whether the UKCAT scores had any predictive value for the later parts of the course, largely delivered via clinical placements. METHODS: Students entering the course in 2007 and who had taken the UKCAT were asked for permission to use their anonymised data in research. The UKCAT scores were incorporated into a database with routine pre-admission socio-demographics and subsequent course performance data. Correlation analysis was followed by hierarchical multivariate linear regression. RESULTS: The original study group comprised 204/254 (80%) of the full entry cohort. With attrition over the five years of the course this fell to 185 (73%) by Year 5. The Verbal Reasoning score and the UKCAT Total score both demonstrated some univariate correlations with clinical knowledge marks, and slightly less with clinical skills. No parts of the UKCAT proved to be an independent predictor of clinical course marks, whereas prior attainment was a highly significant predictor (p <0.001). CONCLUSIONS: This study of one cohort of Nottingham medical students showed that UKCAT scores at admission did not independently predict subsequent performance on the course. Whilst the test adds another dimension to the selection process, its fairness and validity in selecting promising students remains unproven, and requires wider investigation and debate by other schools.
Project description:The high attrition rate among science, technology, engineering, and mathematics (STEM) majors has long been an area of concern for institutions and educational researchers. The transition from introductory to advanced courses has been identified as a particularly "leaky" point along the STEM pipeline, and students who struggle early in an introductory STEM course are predominantly at risk. Peer-tutoring programs offered to all students in a course have been widely found to help STEM students during this critical transition, but hiring a sufficient number of tutors may not be an option for some institutions. As an alternative, this study examines the viability of an optional peer-tutoring program offered to students who are struggling in a large-enrollment, introductory biology course. Struggling students who regularly attended peer tutoring increased exam performance, expert-like perceptions of biology, and course persistence relative to their struggling peers who were not attending the peer-tutoring sessions. The results of this study provide information to instructors who want to design targeted academic assistance for students who are struggling in introductory courses.
Project description:10-15% of students struggle at some point in their medicine course. Risk factors include weaker academic qualifications, male gender, mental illness, UK ethnic minority status, and poor study skills. Recent research on an undergraduate medicine course provided a toolkit to aid early identification of students likely to struggle, who can be targeted by established support and study interventions. The present study sought to extend this work by investigating the number and characteristics of strugglers on a graduate-entry medicine (GEM) programme.A retrospective study of four GEM entry cohorts (2003-6) was carried out. All students who had demonstrated unsatisfactory progress or left prematurely were included. Any information about academic, administrative, personal, or social difficulties, were extracted from their course progress files into a customised database and examined.362 students were admitted to the course, and 53 (14.6%) were identified for the study, of whom 15 (4.1%) did not complete the course. Students in the study group differed from the others in having a higher proportion of 2ii first degrees, and scoring less well on GAMSAT, an aptitude test used for admission. Within the study group, it proved possible to categorise students into the same groups previously reported (struggler throughout, pre-clinical struggler, clinical struggler, health-related struggler, borderline struggler) and to identify the majority using a number of flags for early difficulties. These flags included: missed attendance, unsatisfactory attitude or behaviour, health problems, social/family problems, failure to complete immunity status checks, and attendance at academic progress committee.Problems encountered in a graduate-entry medicine course were comparable to those reported in a corresponding undergraduate programme. A toolkit of academic and non-academic flags of difficulty can be used for early identification of many who will struggle, and could be used to target appropriate support and interventions.
Project description:Understanding course climate is important for improving students' experiences and increasing the likelihood of their persistence in STEM fields. This study presents climate survey results from 523 students taking introductory biology at the University of Michigan. Principal component analysis revealed that a student's climate experience is comprised of five main elements: comfort, school avoidance, relationship to course, academic stress, and discomfort. Of these climate factors, comfort, school avoidance, and relationship to course were significant predictors of course satisfaction, and academic stress was a significant predictor of persistence. The results indicated the importance of a positive climate that is facilitated by the instructor in order to promote a positive student experience. Climate may be an important metric for institutions to track across time and course.
Project description:BACKGROUND: Students who fail to thrive on the Nottingham undergraduate medical course frequently suffer from anxiety, depression or other mental health problems. These difficulties may be the cause, or the result of, academic struggling. Early detection of vulnerable students might direct pastoral care and remedial support to where it is needed. We investigated the use of the short-form General Health Questionnaire (GHQ-12) as a possible screening tool. METHODS: Two consecutive cohorts (2006 and 2007) were invited to complete the GHQ-12. The questionnaire was administered online, during the second semester (after semester 1 exams) for the 2006 cohort and during the first semester for the 2007 cohort. All data were held securely and confidentially. At the end of the course, GHQ scores were examined in relation to course progress. RESULTS: 251 students entered the course in 2006 and 254 in 2007; 164 (65%) and 160 (63%), respectively, completed the GHQ-12. In both cohorts, the study and non-study groups were very similar in terms of pre-admission socio-demographic characteristics and overall course marks. In the 2006 study group, the GHQ Likert score obtained part-way through the first year was negatively correlated with exam marks during Years 1 and 2, but the average exam mark in semester 1 was the sole independent predictor of marks in semester 2 and Year 2. No correlations were found for the 2007 study group but the GHQ score was a weak positive predictor of marks in semester 2, with semester 1 average exam mark again being the strongest predictor. A post-hoc moderated-mediation analysis suggested that significant negative associations of GHQ scores with semester 1 and 2 exams applied only to those who completed the GHQ after their semester 1 exams. Students who were identified as GHQ 'cases' in the 2006 group were statistically less likely to complete the course on time (OR=4.74, p 0.002). There was a non-significant trend in the same direction in the 2007 group. CONCLUSIONS: Results from two cohorts provide insufficient evidence to recommend the routine use of the GHQ-12 as a screening tool. The timing of administration could have a critical influence on the results, and the theoretical and practical implications of this finding are discussed. Low marks in semester 1 examinations seem be the best single indicator of students at risk for subsequent poor performance.
Project description:Doctoral students in science disciplines spend countless hours learning how to conduct cutting-edge research but very little time learning to communicate the nature and significance of their science to people outside their field. To narrow this disparity, we created an unusual course titled Communicating Science for doctoral science trainees at Rutgers University. Our goal was to help students develop an advanced ability to communicate their research clearly and accurately and to emphasize its value and significance to diverse audiences. Course design included classroom instruction supplemented with improvisation, video recordings, and ample opportunity for students to practice and receive immediate, constructive feedback in a supportive environment. A multidisciplinary faculty with expertise in science, education, communication, and theater arts taught this course. PhD students came from diverse scientific disciplines, ranging from biology and chemistry to civil engineering. Students also completed a capstone project in which they worked with a professional in the academic or private sector to explore a possible career aspiration. Assessment was in the form of feedback on students' oral and poster presentations, and written abstracts about their research. Student evaluations and comments about course format and content were mostly positive and also provided input for ways to improve the course. We discovered that the diversity of scientific backgrounds among our students enhanced their ability to learn how to communicate their science to others outside their disciplines. We are leveraging the success of our initial course offering to reach other student and faculty groups at Rutgers.