Dataset Information

Predictive power of UKCAT and other pre-admission measures for performance in a medical school in Glasgow: a cohort study.

ABSTRACT:

Background

The UK Clinical Aptitude Test (UKCAT) and its four subtests are currently used by 24 Medical and Dental Schools in the UK for admissions. This longitudinal study examines the predictive validity of UKCAT for final performance in the undergraduate medical degree programme at one Medical School and compares this with the predictive validity of the selection measures available pre-UKCAT.

Methods

This was a retrospective observational study of one cohort of students, admitted to Glasgow Medical School in 2007. We examined the associations which UKCAT scores, school science grades and pre-admissions interview scores had with performance indicators, particularly final composite scores that determine students' postgraduate training opportunities and overall ranking (Educational Performance Measure - EPM, and Honours and Commendation - H&C). Analyses were conducted both with and without adjustment for potential socio-demographic confounders (gender, age, ethnicity and area deprivation).

Results

Despite its predictive value declining as students progress through the course, UKCAT was associated with the final composite scores. In mutually adjusted analyses (also adjusted for socio-demographic confounders), only UKCAT total showed independent relationships with both EPM (p = 0.005) and H&C (p = 0.004), school science achievements predicted EPM (p = 0.009), and pre-admissions interview score predicted neither. UKCAT showed less socio-demographic variation than did TSS.

Conclusion

UKCAT has a modest predictive power for overall course performance at the University of Glasgow Medical School over and above that of school science achievements or pre-admission interview score and we conclude that UKCAT is the most useful predictor of final ranking.

SUBMITTER: Sartania N

PROVIDER: S-EPMC4063234 | biostudies-literature | 2014 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Predictive power of UKCAT and other pre-admission measures for performance in a medical school in Glasgow: a cohort study.

Sartania Nana N McClure John D JD Sweeting Helen H Browitt Allison A

BMC medical education 20140611

<h4>Background</h4>The UK Clinical Aptitude Test (UKCAT) and its four subtests are currently used by 24 Medical and Dental Schools in the UK for admissions. This longitudinal study examines the predictive validity of UKCAT for final performance in the undergraduate medical degree programme at one Medical School and compares this with the predictive validity of the selection measures available pre-UKCAT.<h4>Methods</h4>This was a retrospective observational study of one cohort of students, admitt ...[more]

PMID: 24919950

Similar Datasets

Project description:The 2-hour long United Kingdom Clinical Aptitude Test (UKCAT) is used by many universities in the United Kingdom as part of their selection process for undergraduate medical and dentistry degrees. We aimed to compare the performance of senior doctors in primary and secondary care and across a range of specialties, in a modified version of the medical school entrance examination-the mUKCAT. Lay people were also included in the study. Despite its widespread use, this is the first study that examines the performance of senior clinicians in the UKCAT.The study used a prospective cross-sectional design. It used mock questions from the UKCAT website to generate an mUKCAT that was anticipated to take 15 minutes to complete. In all, 167 doctors at consultant, general practitioner (GP), or specialty trainee grade and 26 lay people took part.The overall mean mUKCAT score of all participants was 2486 (69.1%). Of the total cohort, 126 (65.3%) scored above our designated threshold of 2368 and were deemed to have passed the mUKCAT. Excluding lay people, 113 (67.7%) of the 167 doctors scored above that threshold. Medical specialty was associated with overall score (P = 0.003), with anesthetists/intensive care physicians scoring highest (n = 20, mean score 2660) and GPs scoring lowest (n = 38, mean score 2302). Academics outperformed nonacademics (mean score of academics, n = 44 vs nonacademics, n = 123: 2750 vs 2406; P < 0.001). Those clinicians in senior management positions scored lower than those in "standard" roles (mean score of senior management, n = 31 vs standard roles, n = 136: 2332 vs 2534, mean difference 202, 95% confidence interval 67-337, P = 0.004).In the situational judgement section, there was no evidence that specialty was associated with score (P = 0.15). Academics exhibited greater situational judgement than their nonacademic colleagues (academics vs nonacademics: 69.8 vs 63.6%; P = 0.01).The majority of senior clinicians passed our mUKCAT. Academics and anesthetists were found to be the best performers, with GPs and those in senior management positions performing the worst.

Project description:ObjectivesFor the first time, this systematic review provides a summary of the literature exploring the relationship between performance in the UK Clinical Aptitude Test (UKCAT) and assessments in undergraduate medical and dental training.DesignIn accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analysis, relevant studies were identified through systematic literature searches. Electronic searches were carried out on EBSCO, EMBASE, Educational Resources Information Centre, SCOPUS, Web of Knowledge. Studies which included the predictive validity of selection criteria including some element of the UKCAT were considered.Results22 papers were identified for inclusion in the study. Four studies describe outcomes from dental programmes with limited results reported. 18 studies reported on relationships between the UKCAT and performance in undergraduate medical training. Of these, 15 studies reported relationships between the UKCAT cognitive tests and undergraduate medical assessments. Weak relationships (r=0.00-0.29) were observed in 14 of these studies; four studies reported some moderate relationships (r=0.30-0.49). The strongest relationships with performance in medical school were observed for the UKCAT total score and the verbal reasoning subtest. Relationships with knowledge-based assessments scores were higher than those for assessments of skills as the outcome. Relationships observed in small (single and double centre studies) were larger than those observed in multicentre studies.ConclusionThe results indicate that UKCAT scores predict performance in medical school assessments. The relationship is generally weak, although noticeably stronger for both the UKCAT total score and the verbal reasoning subtest. There is some evidence that UKCAT continues to predict performance throughout medical school. We recommend more optimal approaches to future studies. This assessment of existing evidence should assist medical/dental schools in their evaluation of selection processes.

Project description:BackgroundAlthough used widely, there is limited evidence of the BioMedical Admissions Test's (BMAT) predictive validity and incremental validity over prior educational attainment (PEA). We investigated BMAT's predictive and incremental validity for performance in two undergraduate medical schools, Imperial College School of Medicine (ICSM), UK, and Lee Kong Chian School of Medicine (LKCMedicine), Singapore. Our secondary goal was to compare the evidence collected with published evidence relating to comparable tools.MethodsThis was a retrospective cohort study of four ICSM (1188 students, entering 2010-2013) and three LKCMedicine cohorts (222 students, 2013-2015). We investigated associations between BMAT Section 1 ('Thinking Skills'), Section 2 ('Scientific Knowledge and Applications') and Section 3a ('Writing Task') scores, with written and clinical assessment performance across all programme years. Incremental validity was investigated over PEA (A-levels) in a subset of ICSM students.ResultsWhen BMAT sections were investigated independently, Section 2 scores predicted performance on all written assessments in both institutions with mainly small effect sizes (standardised coefficient ranges: ICSM: 0.08-0.19; LKCMedicine: 0.22-0.36). Section 1 scores predicted Years 5 and 6 written assessment performance at ICSM (0.09-0.14) but nothing at LKCMedicine. Section 3a scores only predicted Year 5 clinical assessment performance at ICSM with a coefficient <0.1. There were no positive associations with standardised coefficients >0.1 between BMAT performance and clinical assessment performance. Multivariable regressions confirmed that Section 2 scores were the most predictive. We found no clear evidence of incremental validity for any BMAT section scores over A-level grades.DiscussionSchools who wish to assess scientific knowledge independently of A-levels may find BMAT Section 2 useful. Comparison with previous studies indicates that, overall, BMAT seems less useful than comparable tools. Larger scale studies are needed. Broader questions regarding why institutions adopt certain admissions tests, including those with little evidence, need consideration.

Project description:The school-leaving GPA and the time since completion of secondary education are the major criteria for admission to German medical schools. However, the predictive value of the school-leaving grade and the admission delay have not been thoroughly examined since the amendment of the Medical Licensing Regulations and the introduction of reformed curricula in 2002. Detailed information on the prognosis of the different admission groups is also missing.To examine the predictive values of the school-leaving grade and the age at enrolment for academic performance and continuity throughout the reformed medical course.The study includes the central admission groups "GPA-best" and "delayed admission" as well as the primary and secondary local admission groups of three consecutive cohorts. The relationship between the criteria academic performance and continuity and the predictors school-leaving GPA, enrolment age, and admission group affiliation were examined up to the beginning of the final clerkship year.The academic performance and the prolongation of the pre-clinical part of undergraduate training were significantly related to the school-leaving GPA. Conversely, the dropout rate was related to age at enrolment. The students of the GPA-best group and the primary local admission group performed best and had the lowest dropout rates. The students of the delayed admission group and secondary local admission group performed significantly worse. More than 20% of these students dropped out within the pre-clinical course, half of them due to poor academic performance. However, the academic performance of all of the admission groups was highly variable and only about 35% of the students of each group reached the final clerkship year within the regular time.The school-leaving grade and age appear to have different prognostic implications for academic performance and continuity. Both factors have consequences for the delayed admission group. The academic prognosis of the secondary local admission group is as problematic as that of the delayed admission group. Additional admission instruments would be necessary, in order to recognise potentially able applicants independently of their school-leaving grade and to avoid the secondary admission procedure.

Project description:BackgroundKnowledge in natural sciences generally predicts study performance in the first two years of the medical curriculum. In order to reduce delay and dropout in the preclinical years, Hamburg Medical School decided to develop a natural science test (HAM-Nat) for student selection. In the present study, two different approaches to scale construction are presented: a unidimensional scale and a scale composed of three subject specific dimensions. Their psychometric properties and relations to academic success are compared.Methods334 first year medical students of the 2006 cohort responded to 52 multiple choice items from biology, physics, and chemistry. For the construction of scales we generated two random subsamples, one for development and one for validation. In the development sample, unidimensional item sets were extracted from the item pool by means of weighted least squares (WLS) factor analysis, and subsequently fitted to the Rasch model. In the validation sample, the scales were subjected to confirmatory factor analysis and, again, Rasch modelling. The outcome measure was academic success after two years.ResultsAlthough the correlational structure within the item set is weak, a unidimensional scale could be fitted to the Rasch model. However, psychometric properties of this scale deteriorated in the validation sample. A model with three highly correlated subject specific factors performed better. All summary scales predicted academic success with an odds ratio of about 2.0. Prediction was independent of high school grades and there was a slight tendency for prediction to be better in females than in males.ConclusionsA model separating biology, physics, and chemistry into different Rasch scales seems to be more suitable for item bank development than a unidimensional model, even when these scales are highly correlated and enter into a global score. When such a combination scale is used to select the upper quartile of applicants, the proportion of successful completion of the curriculum after two years is expected to rise substantially.

Project description:Objectives: A potential new avenue to address the shortage of country doctors is to change the rules for admission to medical school. We therefore study the link between high-school grade point average and prospective physicians' choice to work in rural areas. To further inform the discussion about rules for admission, we also study the effects of other predictors: a measure of students' attitudes towards risk; whether they waited for their place of study (Wartesemester); whether their parents worked as medical doctors; and whether they have some practical experience in the medical sector. Methods: We conducted two internet surveys in 2012 and 2014. In the first survey, the sample comprised 701 students and in the second, 474 students. In both surveys, we asked students for their regional preferences; in the 2014 survey, we additionally asked students for their first, second, and third preferences among a comprehensive set of specializations, including becoming a general practitioner. In both surveys, we asked students for basic demographic information (age and gender), their parents' occupation, a measure of subjective income expectations, a measure of risk attitudes, and their high-school grade point average (Abiturnote), and First National Boards Examination grade (Physikum). In 2014, we additionally asked for waiting periods (Wartesemester) as well as for prior professional experience in the health-care sector. Results: We find that three factors increase the probability of having a preference for working in a rural area significantly, holding constant all other influences: having a medical doctor among the parents, having worse grades in the high-school grade point average, and being more risk averse. Moreover, we find that those willing to work in the countryside have significantly more experience in the medical sector before admission to medical school. Discussion: Our results suggest that a change in the selection process for medical school may increase the supply of country doctors. Instead of focusing on the high-school grade point average, universities could even more intensely screen for study motivation through interviews or by taking into account students' background, extracurricular activities, or waiting periods.

Project description:ObjectivesTo determine whether the UK Clinical Aptitude Test (UKCAT) adds value to the selection process for school leaver applicants to medical and dental school, and in particular whether UKCAT can reduce the socioeconomic bias known to affect A levels.DesignCohort studySettingApplicants to 23 UK medical and dental schools in 2006.Participants9884 applicants who took the UKCAT in the UK and who achieved at least three passes at A level in their school leaving examinations (53% of all applicants).Main outcome measuresIndependent predictors of obtaining at least AAB at A level and UKCAT scores at or above the 30th centile for the cohort, for the subsections and the entire test.ResultsIndependent predictors of obtaining at least AAB at A level were white ethnicity (odds ratio 1.58, 95% confidence interval 1.41 to 1.77), professional or managerial background (1.39, 1.22 to 1.59), and independent or grammar schooling (2.26, 2.02 to 2.52) (all P<0.001). Independent predictors of achieving UKCAT scores at or above the 30th centile for the whole test were male sex (odd ratio 1.48, 1.32 to 1.66), white ethnicity (2.17, 1.94 to 2.43), professional or managerial background (1.34, 1.17 to 1.54), and independent or grammar schooling (1.91, 1.70 to 2.14) (all P<0.001). One major limitation of the study was that socioeconomic status was not volunteered by approximately 30% of the applicants. Those who withheld socioeconomic status data were significantly different from those who provided that information, which may have caused bias in the analysis.ConclusionsUKCAT was introduced with a high expectation of increasing the diversity and fairness in selection for UK medical and dental schools. This study of a major subgroup of applicants in the first year of operation suggests that it has an inherent favourable bias to men and students from a higher socioeconomic class or independent or grammar schools. However, it does provide a reasonable proxy for A levels in the selection process.

Project description:BackgroundThe medical school admission process is complicated, perhaps reflecting unresolved debates concerning the most important skills necessary to become an ideal physician. The Goldman Medical School at Ben-Gurion University in Israel is known for placing great emphasis on the personal attributes of candidates in addition to their academic excellence. To this end, 1-h consecutive interviews are embedded in the admission process. This study aims to determine whether there is an association between candidates' personal interview ratings and the ratings assigned to these students at the conclusion of their 6th year internal medicine sub-internship.MethodsOur study sample included 136 students who were admitted to the medical school in 2015, and who completed their 6th year internal medicine sub-internship in 2019-2020. Our data were derived from the admissions information for each candidate and from structured interviews concerning medical competence and personal traits, which were completed by medical personnel who were in contact with these students during their clinical rounds.ResultsHigher interview ratings of candidates during the admission process were associated with a higher probability that students would be evaluated as top-rated internists 6 years later (Odds Ratio (OR) = 9.4, p-value = 0.049), independent of gender (OR for male vs female = 0.2, p-value = 0.025) and age (OR = 1.3 per each year, p-value = 0.115). Although significant, the numeric difference in interview rating was relatively small (median 9.5 and 9.4 for top-rated and not top-rated internists, respectively).ConclusionsOur study shows that high personal interview ratings assigned to candidates as part of the medical school admission process are predictive of high performance ratings of students after they complete their 6th year internal medicine sub-internships. These findings demonstrate the value and importance of using semi-structured personal interviews in the medical school admission process.

Dataset Information

Predictive power of UKCAT and other pre-admission measures for performance in a medical school in Glasgow: a cohort study.

Background

Methods

Results

Conclusion

Publications

Predictive power of UKCAT and other pre-admission measures for performance in a medical school in Glasgow: a cohort study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets