- Home
- The UCAT Consortium
- Research
Research
Research Strategy
A core aim of the UCAT Consortium is to improve the evidence base in relation to selection to medicine and dentistry. The organisation continues to support research which has the potential to contribute to the improvement of the test or to inform the use of the test in selection.
Since its inception the UCAT Consortium has made its data available to researchers interested in exploring these areas. More recently these data are hosted by the General Medical Council and can be accessed by approved researchers through the UKMED development. On occasion, the UCAT consortium commissions and funds to researchers to undertake specific projects identified as priorities. This is usually done through invitations to tender to undertake specific projects. Such opportunities are advertised through the UCAT Consortium and wider networks in medical and dental education.
The UCAT Research and Development Group (RDG) guides the research strategy. At the same time the RDG considers issues raised in published research conducted within the UCAT Consortium and beyond.
At registration, the UCAT Consortium collects a significant amount of demographic data regarding its candidates. At the end of each test cycle these data are transferred to the University of Dundee Health Informatics Centre who store and process these data on behalf of the Consortium. Data is stored securely and UCAT commits through its Data Privacy Policy to only undertake studies on fully anonymised data.
If you are undertaking research relating to the UCAT please feel free to email us. We can put you in touch with other researchers active in the field of work.
Published Research
Below is a summary of relevant published work since 2009.
Research published prior to 2019 will refer to UCAT by its previous name: UKCAT.
Exploring performance differences between UCAT candidates who sit standard and extended versions of the test: report for the UCAT Board
July 2024, Lewis W Paton, Paul A Tiffin (Hull York Medical School, University of York)
There has been extensive research published on the UCAT, such as the relationship between demographic variables and performance on the different subtests. However, much of the published research does not distinguish between those who take the UCAT and those who take an extended version. This report aims to inform considerations regarding UCATSEN, as well as future research in the area.
An assessment of the impact of formal preparation activities on performance in the University Clinical Aptitude Test (UCAT): a national study
BMC Medical Education, Published 28 October 2022, Article number 747 (2022), Sanat Kulkarni, Jayne Parry and Alice Sitch
Previous studies have shown performance in the University Clinical Aptitude Test (UCAT) to be associated with measures of candidate socio-economic advantage such as parental occupation and type of school attended. It is possible that access to preparation support and materials may in part explain these associations. In this paper we determine whether use of formal preparation resources is associated with higher UCAT scores and whether differences in use of preparation resources exist between socio-demographic groups.
Situational judgement test performance and subsequent misconduct in medical students
Medical Education 15 March 2022, Emily Sanger, Daniel T. Smith, Adam Troughton and Lewis W. Paton
Situational judgement tests (SJTs) have been widely adopted, internationally, into medical selection. It was hoped that such assessments could identify candidates likely to exhibit future professional behaviours. Understanding how performance on such tests may predict the risk of disciplinary action during medical school would provide evidence for the validity of such SJTs within student selection. It would also inform the implementation of such tests within student recruitment.
UCAT and dental student selection in the UK - what has changed?
British Dental Journal, Published 11 March 2022, Rachel Greatrix and Robert McAndrew
The University Clinical Aptitude Test (UCAT) has been used since 2006 by a consortium of UK medical and dental schools to assist in undergraduate selection. In 2019, UCAT was used by 30 universities (14 dental schools). The aim was to report how UCAT use has changed in undergraduate student selection in the UK.
Can achievement at medical admission tests predict future performance in postgraduate clinical assessments? A UK-based national cohort study
BMJ Open, Published 8 February 2022, Lewis W Paton, I C McManus, Kevin Yet Fong Cheung and Daniel Thomas Smith
To determine whether scores on two undergraduate admissions tests (BioMedical Admissions Test (BMAT) and University Clinical Aptitude Test (UCAT)) predict performance on the postgraduate Membership of the Royal Colleges of Physicians (MRCP) examination, including the clinical examination Practical Assessment of Clinical Examination Skills (PACES).
Research Archives
Assessing the predictive validity of the UCAT—A systematic review and narrative synthesis
Taylor & Francis Online, Published 23 November 2021, Laksha Bala, Stephen Pedder, Amir H. Sam and Celia Brown
https://www.tandfonline.com/doi/full/10.1080/0142159X.2021.1998401
The University Clinical Aptitude Test (UCAT) is an admissions assessment used by a consortium of universities across the UK, Australia, and New Zealand, to aid the selection of applicants to medical and dental degree programmes. The UCAT aims to measure the mental aptitude and professional behaviours required to become successful doctors and dentists. We conducted a systematic review to establish the predictive value of the UCAT across measures of performance at undergraduate and post-graduate levels.
Medical School Selection scores correlate with MRCS Part A performance
British Journal of Surgery, Volume 108, Issue Supplement 7, October 2021, Ricky Ellis, Peter Brennan, Jennifer Cleland, Amanda Lee and Duncan Scrimgeour
https://academic.oup.com/bjs/article/108/Supplement_7/znab362.081
Selection into UK medical school involves a combination of three measures: prior academic attainment, selection tests (e.g. the University Clinical Aptitude Test (UCAT), Biomedical Admissions Test (BMAT), or Graduate Medical School Admissions Test (GAMSAT)) followed by interview. We investigated the predictive power of current UK medical selection tests and measures of prior attainment on success in the Membership of the Royal College of Surgeons (MRCS) examination.
Is the UCAT appropriate for selecting undergraduate medical students?
The Medical Journal of Australia, Published 1 February 2021, David A Powis, Don Munro and Miles R Bore
https://www.mja.com.au/journal/2021/214/2/ucat-appropriate-selecting-undergraduate-medical-students
Non‐academic personal qualities are also desirable in doctors: higher mental abilities, empathy, ethics, creativity.
Medicine is an enduringly popular career choice for several reasons, including the desire to heal the sick and to improve people’s health, but for many, no doubt, also the prospect of guaranteed employment with a satisfactory income and the elevated social status enjoyed by doctors. The net result is that for most medical schools there have been, and still are, far more applicants than places available. This poses an annual selection dilemma.
Does the UKCAT predict performance in medical and dental school? A systematic review
BMJ Open January 2021, Rachel Greatrix, Sandra Nicholson and Susan Anderson
https://bmjopen.bmj.com/content/bmjopen/11/1/e040128.full.pdf
For the first time, this systematic review provides a summary of the literature exploring the relationship between performance in the UK Clinical Aptitude Test (UKCAT) and assessments in undergraduate medical and dental training. The results indicate that UKCAT scores predict performance in medical school assessments. The relationship is generally weak, although noticeably stronger for both the UKCAT total score and the verbal reasoning subtest. There is some evidence that UKCAT continues to predict performance throughout medical school.
The change from UMAT to UCAT for undergraduate medical school applicants: impact on selection outcomes
The Medical Journal of Australia, Published November 2020, Graeme L Horton, Lisa Lampe, Boaz Shuiruf and Wendy Hu
To assess whether the change from the Undergraduate Medical and Health Sciences Admissions Test (UMAT; 1991–2019) to the University Clinical Aptitude Test (UCAT) for the 2020 New South Wales undergraduate medical degree intake was associated with changes in the impact of sex, socio‐economic status and remoteness of residence, and professional coaching upon selection for interview.
UKCAT and medical student selection in the UK - what has changed since 2006?
BMC Medical Education, Article number 292 (2020), Rachel Greatrix and Jonathan Dowell
https://bmcmededuc.biomedcentral.com/articles/10.1186/s12909-020-02214-1
The United Kingdom Clinical Aptitude Test (UKCAT) is an aptitude test used since 2006 within selection processes of a consortium of UK medical and dental schools.
Since 2006, student numbers have increased in medical training and schools now have an increased focus on widening access. A growing evidence base has emerged around medical student selection (Patterson et al., Med Educ 50:36–60, 2016) leading to changes in practice. However, whilst some papers describe local selection processes, there has been no overview of trends in selection processes over time across Universities.
This study reports on how the use of the UKCAT in medical student selection has changed and comments on other changes in selection processes.
Does ‘online confidence’ predict application success and later academic performance in medical school? A UK-based national cohort study
BMJ Open, December 2019; Paul A Tiffin and Lewis W Paton
https://bmjopen.bmj.com/content/9/12/e034437
The UK Clinical Aptitude Test (UKCAT) previously piloted an assessment of ‘online confidence’, where candidates were asked to indicate how confident they were with their answers. This study examines the relationship between these ratings, the odds of receiving an offer to study medicine and subsequent undergraduate academic performance.
Artificial or intelligent? Machine learning and medical selection: possibilities and risks
MedEdPublish 2018, Paul Tiffin and Lewis Paton
https://www.mededpublish.org/manuscripts/2023
Machine learning approaches form the basis of “artificial intelligence” and have been increasingly applied in health services settings. It has been shown that such approaches may produce more accurate predictions in some contexts, compared to conventional statistical approaches, and may also reduce the costs of decision-making through automation. Nevertheless, there are both general limitations to developing and implementing machine learning approaches that must be borne in mind. To date, relatively little research has been published on the potential for machine learning to support personnel selection. Moreover, there are particular challenges and issues that need to be considered if such methods are to be used to support decision-making in medical selection scenarios. This article describes some of these potential advantages and challenges and presents an illustrative example, based on real-world data, related to the selection of medical undergraduates.
Do changing medical admissions practices in the UK impact on who is admitted? An interrupted time series analysis
BMJ Open October 2018; Shona Fielding, Paul Tiffin, Rachel Greatrix, Amanda J Lee, Fiona Patterson, Sandra Nicholson and Jennifer Cleland
https://bmjopen.bmj.com/content/8/10/e023274
Medical admissions must balance two potentially competing missions: to select those who will be successful medical students and clinicians and to increase the diversity of the medical school population and workforce. Many countries address this dilemma by reducing the heavy reliance on prior educational attainment, complementing this with other selection tools. However, evidence to what extent this shift in practice has actually widened access is conflicting. This study sought to analyse if changes in medical school selection processes significantly impacted on the composition of the student population.
Exploring the Relationship between the UKCAT Situational Judgement Test and the Multiple Mini Interview
Published August 2018, Adrian Husbands, Jonathan Dowell, Matthew Homer, Robert McAndrew and Rachel Greatrix
In 2013 the UKCAT introduced a Situational Judgement Test (SJT) alongside its four cognitive subtests, which intended to provide an assessment of non-cognitive traits including integrity, perspective taking & team involvement. Before the introduction of SJTs, MMIs were the only widely-used selection tool to objectively evaluate a range of personal qualities during medical & dental student selection. This study investigated associations between the UKCAT cognitive component, UKCAT SJT and MMI scores among candidates at member institutions using the tools in selection.
Analysis of Confidence Rating Pilot Data: Executive Summary for the UKCAT Board
July 2018, Paul A Tiffin and Lewis W Paton
Self-confidence may be the best ‘non-cognitive’ predictor of future academic performance, moreover, it is important that future doctors are neither over, nor under-confident in their own abilities if they are to practice medicine safely and effectively. From 2013 to 2016 a confidence rating was piloted within the Decision Analysis section of the UKCAT. This research analysed these data to explore whether ‘online confidence’ was predictive of either the odds of success at application, or academic performance during the first two years of medical school.
What is the effect of secondary (high) schooling on subsequent medical school performance? A national, UK-based, cohort study
BMJ Open January 2018; 8:5 Lazaro M Mwandigha, Paul A Tiffin, Lewis W Paton, Adetayo S Kasim and Jan R Böhnke
http://bmjopen.bmj.com/content/8/5/e020291
University academic achievement may be inversely related to the performance of the secondary (high) school an entrant attended. Some medical schools already offer ‘grade discounts’ to applicants from less well-performing schools. However, evidence to guide such policies is lacking. This study analyses a national dataset in order to understand the relationship between the two main predictors of medical school admission in the UK (prior educational attainment and performance on the UKCAT) and subsequent undergraduate knowledge and skills-related outcomes analysed separately.
The ability of ‘non-cognitive’ traits to predict undergraduate performance in medical schools: a national linkage study
BMC Medical Education 2018 18:93. Gabrielle M. Finn, Lazaro Mwandigha, Lewis W. Paton and Paul A. Tiffin
https://doi.org/10.1186/s12909-018-1201-7
In addition to the evaluation of educational attainment and intellectual ability there has been interest in the potential to select medical school applicants on non-academic qualities. Consequently, a battery of self-report measures concerned with assessing ‘non-cognitive’ traits was piloted as part of the UK Clinical Aptitude Test (UKCAT) administration to evaluate their potential to be used in selection.
The relationship between UKCAT scores and Finals exam performance for widening access and traditional entry students
Srikathirkamanathan K, Curtis SA and McManus BN, University of Southampton
This retrospective, cohort study at the University of Southampton examined the correlation between the UKCAT scores and the total Finals examination performance measure and its component examinations for students who entered in 2009. The significant correlations between the UKCAT score and the Verbal Reasoning score and some assessment outcomes supported the use of UKCAT in the admissions process.
The Relationship Between School Type and Academic Performance at Medical School: a National Multi-cohort Study
BMJ Open August 2017; 7:8, B Kumwenda, J Cleland, K Walker, A Lee, R Greatrix
http://bmjopen.bmj.com/content/7/8/e016291
Differential attainment in school examinations is one of the barriers to increasing student diversity in medicine. However, studies on the predictive validity of prior academic achievement and educational performance at medical school are contradictory, possibly due to single-site studies or studies which focus only on early years’ performance. To address these gaps, this study examined the relationship between sociodemographic factors, including school type and average educational performance throughout medical school across a large number of diverse medical programmes.
The Predictive Validity of a Text-Based Situational Judgment Test in Undergraduate Medical and Dental School Admissions
Academic Medicine (2017), Patterson, F., Cousans, F. et al.
http://eprints.whiterose.ac.uk/114279/1/The_Predictive_Validity_of_a_Text_Based.98258_1_.pdf
Situational judgment tests (SJTs) can be used to assess the nonacademic attributes necessary for medical and dental trainees to become successful practitioners. Evidence for SJTs’ predictive validity, however, relates predominantly to selection in postgraduate settings or using video-based SJTs at the undergraduate level; it may not be directly transferable to text-based SJTs in undergraduate medical and dental school selection. This preliminary study aimed to address these gaps by assessing the validity of the UK Clinical Aptitude Test (UKCAT) text-based SJT.
Report to the UKCAT board: Enhancing the scoring of the situation judgement test component of the UKCAT; reanalysis of the validity pilot study data
Paul Tiffin & Lewis Paton, University of York, October 2016 (Summary report produced March 2017)
Previous work analysing item level data from the situation judgement tests (SJTs) suggest that reliability could be potentially increased by using an unweighted scoring system, selecting a subset of items that relate strongly to the main dimensional being measured and exploring scoring patterns using item response theory (IRT) approaches. However, it was unclear whether increasing the reliability of the test would automatically increase validity, as it may have been possible that important test content could be lost taking this approach. The original validity pilot study data provided an opportunity to evaluate competing scoring approaches in respects of both reliability and validity.
Exploring the validity of the 2013 UKCAT SJT- prediction of undergraduate performance in the first year of medical school: Summary Version of Report
Paul A. Tiffin & Lewis W. Paton, Health Sciences, University of York, March 2017
Previously it has been shown that the scores from some forms of the UKCAT 2013 SJT predict tutor ratings in medical students. In addition, the cognitive scale scores of the UKCAT tend to predict undergraduate performance on both theory and skills-based assessments in medical school. In these analyses we model the relationship between such undergraduate academic performance measures and the 2013 SJT scores.
Do personality traits assessed on medical school admission predict exit performance? A UK-wide longitudinal cohort study
Adv in Health Sci Educ (2016), MacKenzie, R.K., Dowell, J., Ayansina, D. et al.
Traditional methods of assessing personality traits in medical school selection have been heavily criticised. To address this at the point of selection, “non-cognitive” tests were included in the UKCAT, the most widely-used aptitude test in UK medical education. This study examined the predictive validity of these non-cognitive traits with performance during and on exit from medical school by sampling all students graduating in 2013 from the 30 UKCAT consortium medical schools.
Predictive validity of the UKCAT for medical school undergraduate performance: a national prospective cohort study
BMC Medicine 2016 14:140, Paul A. Tiffin, Lazaro M. Mwandigha, Lewis W. Paton, H. Hesselgreaves, John C. McLachlan, Gabrielle M. Finn and Adetayo S. Kasim
http://bmcmedicine.biomedcentral.com/articles/10.1186/s12916-016-0682-7
The UKCAT has been shown to have a modest but statistically significant ability to predict aspects of academic performance throughout medical school. Previously, this ability has been shown to be incremental to conventional measures of educational performance for the first year of medical school. This study evaluates whether this predictive ability extends throughout the whole of undergraduate medical study and explores the potential impact of using the test as a selection screening tool.
Does the UKCAT predict performance on exit from medical school? A national cohort study
BMJ Open 2016; 6 R K MacKenzie, J A Cleland, D Ayansina, S Nicholson
http://bmjopen.bmj.com/content/6/10/e011313.full
Most UK medical programmes use aptitude tests during student selection, but large-scale studies of predictive validity are rare. This study assesses the UKCAT, and 4 of its subscales, along with individual and contextual socioeconomic background factors, as predictors of performance during, and on exit from, medical school.
Longitudinal assessment of the impact of the use of the UK clinical aptitude test for medical student selection
Medical Education 2016 50:10, Jonathan Mathers, Alice Sitch, Jayne Parry
http://onlinelibrary.wiley.com/doi/10.1111/medu.13082/abstract
This paper aims to provide the first longitudinal analyses assessing the impact of the different uses of UKCAT on making offers to applicants with different socio-demographic characteristics. Multilevel regression was used to model the outcome of applications to UK medical schools during the period 2004–2011. The authors’ findings demonstrate attenuation of the advantage of being female but no changes in admission rates based on White ethnicity, higher social class and selective schooling. In view of this, the utility of the UKCAT as a means to widen access to medical schools among non-White and less advantaged applicants remains unproven.
Do differentials in access to advice and support at UK schools on preparation for the UK Clinical Aptitude Test disadvantage some candidate groups?
Paul Lambe, Rachel Greatrix, Keith Milburn, Jon Dowell, David Bristow
The UKCAT is part of the selection process of the majority of UK medical and dental schools. However, a survey of applicants to a single UK medical school indicated that differentials in access to advice and support on test preparation may disadvantage some candidate groups. To assess the generalizability of this finding and extend understanding, a survey of the 2012 UKCAT candidate cohort was conducted.
Candidates’ views on the UK Clinical Aptitude Test: Results from the Post 2012 UKCAT Survey
Paul Lambe, Rachel Greatrix, Keith Milburn, Jon Dowell, David Bristow
Aptitude test are used internationally with the aims of complementing existing selection tools, improving the fairness of selection and widening access to the study of medicine. The UKCAT is employed by the majority of UK medical and dental schools. Recent studies have raised concern about the lack of transparency about test usage in the selection process and the impact this may have on widening access. Along with examining candidates’ views about the test, this study examines the question whether candidates’ understanding of how universities use the UKCAT in the selection process influences application decisions, and, what impact this may have on widening access, given UKCAT score achieved and candidate background.
Widening access in selection using situational judgement tests: evidence from the UKCAT
Medical Education 2016 50:6, Filip Lievens, Fiona Patterson, Jan Corstjens, Stuart Martin and Sandra Nicholson
http://onlinelibrary.wiley.com/doi/10.1111/medu.13060/full
Widening access promotes student diversity and the appropriate representation of all demographic groups. This study aims to examine diversity-related benefits of the use of situational judgement tests (SJTs) in the UK Clinical Aptitude Test (UKCAT) in terms of three demographic variables: (i) socio-economic status (SES); (ii) ethnicity, and (iii) gender.
Does a selection interview predict year 1 performance in dental school?
European Journal of Dental Education 2016, R. McAndrew, J. Ellis and R. A. Valentine
http://onlinelibrary.wiley.com/enhanced/doi/10.1111/eje.12188
This study was undertaken in order to determine whether different interview methods (Cardiff with a multiple mini interview and Newcastle with a more traditional interview process) along with other components used in selection predicted academic performance in students.
A Cross-sectional Study of Current Doctors’ Performance in a Modified Version of a Medical School Admission Aptitude Test
Medicine (Baltimore) 2016 95:18, James P. Blackmur, Nazir I. Lone, Oliver D. Stone, David J. Webb, and Neeraj Dhaun
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4863766/
This study aimed to compare the performance of senior doctors in primary and secondary care and across a range of specialties, in a modified version of the medical school entrance examination—the mUKCAT. Lay people were also included in the study. Despite its widespread use, this is the first study that examines the performance of senior clinicians in the UKCAT.
UKCAT SJT: a study to explore validation methodology and early findings
Fiona Patterson, Stuart Martin – download report
The paper summarises the initial evaluation of the SJT undertaken by the Work Psychology Group in 2014. The evaluation of the methodology and initial outcomes informed the development of a further study which took place the following year. Results from the second study are being prepared for publication.
Fair access to medicine? Retrospective analysis of UK medical schools application data 2009-2012 using three measures of socioeconomic status
BMC Medical Education 2016 16:11 Kathryn Steven, Jon Dowell, Cathy Jackson and Bruce Guthrie
http://bmcmededuc.biomedcentral.com/articles/10.1186/s12909-016-0536-1
This study used UKCAT data to examine variation in socioeconomic status (SES) of applicants to study medicine and applicants with an accepted offer from a medical school. The authors conclude that regardless of the measure, those from less affluent backgrounds are less likely to apply and less likely to gain an accepted offer to study medicine. Previously unreported variation between UK countries and between medical schools warrants further investigation.
The UKCAT research panel commissioned item-level analyses of both the cognitive subtests and the situational judgement tests. These two reports are published to provide additional information regarding the 'measurement model' of the UKCAT for researchers.
Understanding the Dimensionality and Reliability of the Cognitive Scales of the UK Clinical Aptitude test (UKCAT): Summary Version of the Report
Dr Paul Tiffin, December 2013 – download report
Understanding the measurement model of the UKCAT Situational Judgement Test: Summary Report
Paul Tiffin & Madeline Carter, May 2015 – download report
Predictive value of the admissions process and the UK Clinical Aptitude Test in a graduate-entry dental school
British Dental Journal 2015, 218: 687-689 J. I. Foley and K. Hijazi
http://informahealthcare.com/doi/full/10.3109/0142159X.2015.1009023
The study aims to assess the association between admissions performance and the UKCAT, and subsequent achievement within a graduate-entry dental school. It was conducted at the University of Aberdeen Dental School between 2010 and 2014. Student demographics, pre-admission scores (PAS), Universities and Colleges Admissions Service (UCAS) tariffs, multiple mini-interview (MMI) grades, UKCAT scores and percentiles were correlated with academic performance reported as the University Common Assessment Scale (0–20). This study suggests that student age, candidate performance at MMI and the UKCAT might be a predictor of academic achievement for graduate-entry dental students.
Predictors of professional behaviour and academic outcomes in a UK medical school: A longitudinal cohort study
Medical Teacher 2015, 1-13 Jane Adam, Miles Bore, Roy Childs, Jason Dunn, Jean Mckendree, Don Munro, and David Powis
http://informahealthcare.com/doi/full/10.3109/0142159X.2015.1009023
The study followed a complete intake of students throughout medical school to determine whether particular variables at the point of selection might distinguish those more likely to become satisfactory professional doctors. HYMS academic score, some UKCAT subtest scores and the total UKCAT score, and some non-cognitive tests completed at the outset of studies, together predicted outcomes most comprehensively. Tutor evaluation of students early in the course also identified the more and less successful students in the three domains of academic, clinical and professional performance. These results may be helpful in informing the future development of selection tools.
Predictive power of UKCAT and other pre-admission measures for performance in a medical school in Glasgow: a cohort study
BMC Medical Education 2014, 14:116 Nana Sartania, John D McClure, Helen Sweeting and Allison Browitt
http://www.biomedcentral.com/1472-6920/14/116/abstract
Results from the study shows that UKCAT has a modest predictive power for overall course performance at the University of Glasgow Medical School over and above that of school science achievements or pre-admission interview score. The researchers go on to conclude that UKCAT is the most useful predictor of final ranking.
Predictive validity of the UK clinical aptitude test in the final years of medical school: a prospective cohort study
BMC Medical Education 2014, 14:88 Adrian Husbands, Alistair Mathieson, Jonathan Dowell, Jennifer Cleland, Rhoda MacKenzie
http://www.biomedcentral.com/1472-6920/14/88/abstract
The study concludes that neither UCAS form nor interview scores were statistically significant predictors of examination performance but that conversely, the UKCAT yielded statistically significant validity coefficients in four of five assessments investigated. Multiple regression analysis showed the UKCAT made a statistically significant unique contribution to variance in examination performance in the senior years.
Comparison of the sensitivity of the UKCAT and A Levels to sociodemographic characteristics: a national study
BMC Medical Education 2014, 14:7 Paul A Tiffin, John C McLachlan, Lisa Webster and Sandra Nicholson
http://www.biomedcentral.com/1472-6920/14/7/abstract
Researchers concluded that most of the sociodemographic factors that predict A level attainment also predict UKCAT performance. However, compared to A levels, males and those speaking English as a first language perform better on UKCAT. The findings suggest that UKCAT scores may be more influenced by sex and less sensitive to school type compared to A levels. These factors must be considered by institutions utilising the UKCAT as a component of the medical and dental school selection process.
The UKCAT-12 study: Educational attainment, aptitude test performance, demographic and socio economic contextual factors as predictors of first year outcome in a cross-sectional collaborative study of twelve UK medical schools
BMC Medicine 2013, 11:244 IC McManus, Chris Dewberry, Sandra Nicholson and Jonathan S Dowell
http://www.biomedcentral.com/1741-7015/11/244
This study looked at 4,811 students in 12 UK medical schools who took the UKCAT from 2006 to 2008.
Researchers concluded that UKCAT scores and educational attainment measures were significant predictors of outcome. The incremental validity of UKCAT taking educational attainment into account was significant, but small. Medical school performance was also affected by a contextual measure of secondary schooling, students from secondary schools with greater average attainment at A-level performing less well.
This collaborative study in 12 medical schools shows the power of large-scale studies of medical education for answering previously unanswerable but important questions about medical student selection, education and training.
Construct-level predictive validity of educational attainment and intellectual aptitude tests in medical student selection: Meta-regression of six UK longitudinal studies
BMC Medicine 2013, 11:243 I C McManus, Chris Dewberry, Sandra Nicholson, Jonathan S Dowell, Katherine Woolf and Henry WW Potts
http://www.biomedcentral.com/1741-7015/11/243
Construct-level predictive validities were calculated in six cohort studies of medical student selection and training (student entry, 1972 to 2009) for a range of predictors, including A-levels, General Certificates of Secondary Education (GCSEs)/O-levels, and aptitude tests (AH5 and UK Clinical Aptitude Test (UKCAT)). Outcomes included undergraduate basic medical science and finals assessments, as well as postgraduate measures of Membership of the Royal Colleges of Physicians of the United Kingdom (MRCP(UK)) performance and entry in the Specialist Register.
Educational attainment has strong CLPVs for undergraduate and postgraduate performance, accounting for perhaps 65% of true variance in first year performance. Such CLPVs justify the use of educational attainment measure in selection, but also raise a key theoretical question concerning the remaining 35% of variance. The researchers suggest that some variance probably relates to factors which are unpredictable at selection, such as illness or other life events, but some is probably also associated with factors such as personality, motivation or study skills.
Forecasting the impact of the mode of use of the UKCAT on medical and dental school entrant demographics
Paul Tiffin, Lisa Webster & John McLachlan October 2013
The results of previous analyses suggest that the manner in which the UKCAT is used in the admissions process may influence the demographic characteristics of entrants. These findings were used to develop forecasts of what may happen to the demographics of the medical and dental students if medical and dental schools were to change their usage style of the test. Results suggest that if consortium medical schools currently using a ‘borderline’ or ‘factor’ approach to the UKCAT scores switched to using the test results as a ‘threshold’ for interview or offer there would be modest increases (roughly 5-7%) in the proportion of males admitted to ‘standard entry’ medical courses. In addition, if medical schools currently using a ‘factor’ approach to the UKCAT scores changed to a ‘threshold’ approach there may be an appreciable increase in the proportion of ex-state school students admitted to ‘standard entry’ medical courses (roughly in the order of 10%). If consortium dental schools currently using a ‘factor’ approach to the UKCAT scores switched to a ‘threshold’ approach then this may result in an appreciable increase in the proportion of ex-state school students admitted to ‘standard entry’ dental courses, roughly in the order of 10%.
Download the medicine report Download the dental report
The UKCAT-12 Study Technical Report Educational attainment, aptitude test performance, demographic and socio-economic contextual factors as predictors of first year outcome in twelve UK medical schools
IC McManus, C Dewberry, S Nicholson, J Dowell
The following Technical Report was previously submitted to the UKCAT Consortium to inform the Consortium of progress in analyzing the UKCAT-12 datasets. The full report is now being released, unchanged, to coincide with and to supplement the publication in BMC Medicine of a group of three papers, two of which analyse the UKCAT-12 data in more detail. The Report provides more details on some of the analyses and may be of use to those interested in UKCAT. It should be emphasized that the Report was a working document, that none of the conclusions were final, and indeed some of the conclusions and the methods on which they were based have been updated, in some cases as a result of minor errors having been found. The reader should therefore interpret the findings in that context and in conjunction with the three BMC Medicine papers.
Socioeconomic status of applicants to UKCAT Consortium Medical Schools 2009 – 2012
Published on the Medical Schools Council website May 2013 Lynda Cochrane, David Ridley, Bruce Guthrie, Jon Dowell
http://www.medschools.ac.uk/Publications/Pages/UKCAT_socioeconomic_analysis_2013.aspx
This analysis examines the socioeconomic status of applicants to the 26 medical schools in the UKCAT consortium. It assesses the impact of increased fees on the socioeconomic profile of applicants to medicine. It was produced by the Health Informatics Centre, University of Dundee, on behalf of the Medical Schools Council.
The UK clinical aptitude test and clinical course performance at Nottingham: a prospective cohort study
BMC Medical Education 2013, 13:32 Janet Yates and David James
http://www.biomedcentral.com/1472-6920/13/32
The study built on earlier work, investigating whether 2006 UKCAT results predicted perfomance in the clinical components of the medical programme at Nottingham. The authors report that verbal reasoning and the UKCAT total score show modest correlation with clinical course marks. The authors go on to conclude that these are significantly less strong than the predictive ability of performance earlier in the programme.
Can personal qualities of medical students predict in-course examination success and professional behaviour? An exploratory prospective cohort study
BMC Medical Education 2012, 12:69 Jane Adam, Miles Bore, Jean McKendree, Don Munro and David Powis
http://www.biomedcentral.com/1472-6920/12/69/abstract
This study shows numerous significant relationships between both cognitive and non-cognitive test scores, academic examination scores and indicators of professional behaviours in medical students. This suggests that measurement of non-cognitive personal qualities in applicants to medical school could make a useful contribution to selection and admission decisions. Further research is required in larger representative groups, and with more refined predictor measures and behavioural assessment methods, to establish beyond doubt the incremental validity of such measures over conventional cognitive assessments.
The UK Clinical Aptitude Test: Is it a fair test for selecting medical students?
Medical Teacher posted on-line May 2012 Paul Lambe, Catherine Waters, David Bristow
http://informahealthcare.com/doi/full/10.3109/0142159X.2012.687482
The aim of this study was to determine if differences in: access to support and advice, in modes of preparation, type of school/college attended, level of achievement in mathematics, gender and age influence candidate performance in the UKCAT and unfairly advantage some candidates over others. The study was carried out using questionnaires of applicants to study on an undergraduate medical degree course. Researchers found that differentials in access to support and advice, in modes of preparation, type of school/college attended, in level of achievement in mathematics, gender and age were found to be associated with candidate performance in the UKCAT. The findings imply that the UKCAT may disadvantage some candidate groups and that this inequity would likely be improved if tutors and career advisors in schools and colleges were more informed about the UKCAT and able to offer appropriate advice on preparation for the test.
Widening access to UK medical education for under-represented socioeconomic groups: modelling the impact of the UKCAT in the 2009 cohort
BMJ 2012;344:e1805 Paul A Tiffin, Jonathan S Dowell, John C McLachlan
http://www.bmj.com/content/344/bmj.e1805
The authors set out to determine whether the use of the UK clinical aptitude test (UKCAT) in the medical schools admissions process reduces the relative disadvantage encountered by certain sociodemographic groups. They were able to analyse detailed candidate and admissions data and look at how and how strongly different medical schools used the test. They concluded that the use of the UKCAT may lead to more equitable provision of offers to those applying to medical school from under-represented sociodemographic groups. This may translate into higher numbers of some, but not all, relatively disadvantaged students entering the UK medical profession.
Can the UKCAT select suitable candidates for interview?
Medical Education 2011: 45: 1041–1047 Rebecca Turner, Sandra Nicholson
http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2923.2011.03993.x/abstract
This paper examines current selection practices and questions the role that the UKCAT may take, focussing in particular on whether UKCAT can select suitable candidates for interview. The rejection rate before interview of candidates with low UKCAT scores was 2.7 times that of candidates with high UKCAT scores. However, no relationship between overall UKCAT score and overall interview score existed within a pre-selected cohort of applicants with high UKCAT scores. The paper concluded that the UKCAT can facilitate the independent selection of appropriate candidates for interview but that it is not predictive of success at interview.
A mixed-methods study identifying and exploring medical students’ views of the UKCAT
Medical Teacher 2011 Vol. 33, No. 3 , Pages 244-249 J.A. Cleland, F.H. French, P.W. Johnston
http://informahealthcare.com/doi/full/10.3109/0142159X.2011.557753
This was a mixed-methods study using a paper-based survey and focus groups with first year medical students in Scotland in 2009–2010. Questionnaire data were analysed using SPSS, focus group data using framework analysis. The UKCAT was viewed unfavourably by first year medical students completing it pre-admission. These negative views seem due to concern as to the use of UKCAT data, and the fairness of the test. The authors conclude that more evidence as to validity and fairness of the UKCAT and how it is used in practice is required.
Predictive validity of the personal qualities assessment for selection of medical students in Scotland
Medical Teacher 2011, Volume 33 Jon Dowell, Mary Ann Lumsden, David Powis, Don Munro, Miles Bore, Biokanyo Makubate, Ben Kumwenda
The Personal Qualities Assessment (PQA) has been used within the non-cognitive sub-test of the UKCAT, having been developed to enhance medical student selection by measuring a range of non-cognitive attributes in applicants to medical school. This study pre-dates the introduction of the UKCAT with applicants to the five Scottish medical schools piloting the PQA in 2001 and 2002. The study aimed to evaluate the predictive validity of PQA through a longitudinal cohort study in which PQA scores were compared with senior year medical school performance. The most significant finding was that students identified by PQA as ‘not extreme’ on the two personal characteristics scales performed better in an OSCE measure of clinical performance.
Use of UKCAT scores in student selection by UK medical schools, 2006-2010
BMC Medical Education 2011, 11:98 Jane Adam, Jon Dowell, Rachel Greatrix
http://www.biomedcentral.com/1472-6920/11/98/abstract
This paper summarises the way medical schools have used the test since its inception in 2006, categorises that use and demonstrates trends in use over time. Annual telephone interviews were conducted with UKCAT Consortium medical schools. The paper reports that four ways of using the test results have emerged (Borderline, Factor, Threshold and Rescue methods) with many schools using more than one method. The considerable variation in how medical schools use UKCAT supports the wish of the Consortium that applicants are clearly informed about how the test will be used in order that they can make best use of their limited number of applications.
Comparison of A-level and UKCAT performance in students applying to UK medical and dental schools in 2006: cohort study
BMJ 2010;340:bmj.c478 David James, Janet Yates, Sandra Nicholson
http://www.bmj.com/content/340/bmj.c478.full
The object of this study was to determine whether the UK Clinical Aptitude Test (UKCAT) adds value to medical and dental school, and in particular whether UKCAT can reduce the socioeconomic bias known to affect A levels. The study suggests that the UKCAT has an inherent favourable bias to men and students from a higher socioeconomic class or independent or grammar schools. At the same time however, it does provide a reasonable proxy for A levels in the selection process.
The value of the UKCAT in predicting pre-clinical performance: a prospective cohort study at Nottingham medical school
BMC Medical Education 2010 10;55 Janet Yates, David James
http://www.biomedcentral.com/1472-6920/10/55
The study aimed to determine whether UKCAT scores predict performance during the first two years of the 5-year undergraduate medical course at Nottingham. This limited study suggested that the predictive value of the UKCAT, particularly the total score, is low although section scores may predict success in specific types of course assessment.
Has the UK Clinical Aptitude Test improved medical student selection?
Medical Education 2010: 44: 1069 – 1076 Sarah R Wright, Philip M Bradley
http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2923.2010.03792.x/abstract
The study aimed to determine whether the UKCAT had made any improvements to the way medical students are selected. The analysis studied the ability of previous school type and gender to predict UKCAT, personal statement or interview scores in two cohorts of accepted students. The results demonstrated that previous school type was not a significant predictor of either interview or UKCAT scores amongst students who had been accepted onto the programme. UKCAT scores were significant predictors of knowledge examination performance for all but one examination administered in the first two years of Newcastle Medical School whereas interview scores are not.
A comparison of the UKCAT with a traditional admission selection process
Informahealthcare 2009, Vol. 31, No. 11 , Pages 1018-1023 Nishan Fernando, Jennifer Clelland, Kathryn Greaves, Hamish Mckenzie
http://informahealthcare.com/doi/abs/10.3109/01421590802520923?2
The aim of this study was to compare candidate UKCAT performance with medical student selection outcomes at the University of Aberdeen. Results from this study indicate that UKCAT scores show weak correlation with success in Aberdeen’s medical admissions process indicating that the UKCAT examines different traits.
Does the UKCAT predict Year 1 performance in medical school?
Medical Education 2009: 43: 1203 – 1209 Bonnie Lynch, Rhoda MacKenzie, Jon Dowell, Jennifer Clelland, Gordon Prescott
http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2923.2009.03535.x/abstract
This study set out to identify whether UKCAT total score and subtest scores predict Year 1 outcomes at the Universities of Aberdeen and Dundee. UKCAT scores did not predict Year 1 performance at the two medical schools suggesting that studies of UKCAT’s ability to predict outcomes in later years of medical school and in subsequent qualification and practice as a doctor are indispensable in determining whether the use of the test as a selection tool is justified.
