Applications
Title Details Summary Lead Contact
UKMEDP001
Predicting Fitness to Practise issues from admission profiles in UK medical school entrants

Approved on 15 October 2015 UKMED Advisory Board

Published in BMC Medical Education in April 2018 as "Predictors of fitness to practise declarations in UK medical undergraduates"
The selection of medical students is known to be a complex and ‘evidence light’ area with a wide variety of approaches exercised. Although it is established that cognitive and educational performance tends to predict academic performance, especially in the early years of medical school, less is known about the role of personal attributes and the predictors of later problems that may impair Fitness to Practise (FtP). In recent years the GMC has required significant FtP concerns to be reported prior to provisional registration of UK graduates and are now over 300 per year are now considered, with rejection an occasional outcome (0-2/year). This is clearly problematic for the few individuals concerned but there is also a suspicion that those allowed to graduate may be ‘at risk’ of future professional behavioural issues arising. This is largely based on case-control studies conducted by Papadakis (2004) in the USA, who found medical school concerns were associated with future misconduct. A more recent paper by Norman(2015) has questioned the feasibility of this approach. The holy grail of selection is avoiding recruiting ‘problem students’ in the first place but there is no evidence that could be achieved, let alone justified. The advent of the UK Medical Education Database (UKMED) and the possibility of matching a wide range of pre-admissions metrics with outcome markers such as progression through medical school and provisional registration data offers a first opportunity to explore the issue from a modelling perspective and inform selection practice. Graduate cohorts from 2013 and 2014 offer over 12 000 individuals with matched data and approximately 800 reported FtP concerns recorded by the GMC. Around 8 000 of these should include a range of non-academic measures piloted within the UKCAT test in 2007 offering a unique cohort for study and a sufficient sample to merit exploratory analysis. It is recognized that this data is highly sensitive and that great care would be required to ensure individuals could not be identified. This may require some blunting of the available data, such as categorising the most severe offences broadly enough to avoid highlighting the 0-5 excluded within these cohorts too clearly. However, it is suggested that, in outline, this can be achieved using the approach described and therefore UKMED has an opportunity and indeed responsibility to conduct pioneering research on such a critical area of selection practice. There is also the opportunity to follow these cohorts on into practice and to pilot methodologies for a prospective cohort study to that may include performance measures such as ARCP progression and FtP events.

References

  1. Papadakis M, Hodgson C, Teherani A,Kohatsu N. 2004 Unprofessional Behavior in Medical School Is Associated with Subsequent Disciplinary Action by a State Medical Board. Acad Med 79: 244-249. This key cardinal paper reported a case control study in which 68 doctors sanctioned by the Medical Board of California were found to be twice as likely to have concerns regarding professionalism highlighted in their medical school records.
  2. Norman G. Identifying the bad apples. 2015Adv Health Sci EducTheory Pract. 20(2):299-303. Norman challenges the case made by Powis and others regarding non-academic attributes and models the sensitivity and specificity of tests required to screen out doctors who might later fall foul of FtP concerns.
  3. Powis D. 2015. Selecting medical students: An unresolved challenge. Medteach 37: 252-260 (doi:10.3109/0142159X.2014.993600) This review and analysis piece presents an argument for moving beyond selecting for academic excellence by ‘selecting out’ for undesirable non-academic attributes. Amongst others it suggests the Personal Qualities Assessment (as piloted by UKCAT) has potential in this area.
  4. GMC 2012. Report to Undergraduate Board. (Accessed28/7/2105) This report includes the number and type of declarations made by applicants for Provisional registration in the UK.
  5. GMC. 2007 Medical students: professional behaviour and fitness to practice. In conjunction with Maintaining Good Medical Practice (2007) this student focused guidelines describes the expectation of medical students and forms the basis of medical schools approach to Fitness to Practice.

Doctor Paul Tiffin
p.a.tiffin@dur.ac.uk
UKMEDP002
What has been the impact of accelerated graduate-entry medicine courses in terms of educational and sociodemographic profile, success at medical school, completion of Foundation training, and specialty entry?

Approved on 15 October 2015 UKMED Advisory Board

Published in BMC Medical Education in November 2018 as "Impact of accelerated, graduate-entry medicine courses: a comparison of profile, success, and specialty destination between graduate entrants to accelerated or standard medicine courses in UK"
A proportion of each annual intake to medical school already have a first degree in a different subject. Prior to 2000 these graduate entrants studied alongside school leavers in the existing UK five or six year medicine courses. Since then, around 800 graduates annually have entered the fifteen UK graduate-entry 4-year accelerated programmes as well as a smaller number who have continued to join the 5/6 year programmes. The profile of graduate-entrants has also been markedly different from undergraduate programmes in terms of age and subject background: nine of the graduate entry courses admitting students with degrees in non-science subjects. Older research reported (e.g. James & Chilvers, 2001; Wilkinson et al, 2004) that graduates had a number of advantages in terms of attainment and progress at medical school compared to younger entrants with only secondary educational qualifications. However, it was unclear what exactly might be responsible for these advantages. More recent research (McManus et al, 2013) analysed the year 1 performance of entrants to twelve UK medical schools in terms of potential predictors of their attainment. In summary, that study confirmed the strong relationship between A-level performance and year 1 medical school assessments, but it also demonstrated weaker, but incremental predictive validity for a number of other pre-entry variables: these included GCSE performance, scores on UKCAT, and age>21 (who were most likely graduates). Demographic variables were also influential: in particular, men and students from non-white UK ethnic minority communities performed more poorly. Several studies of attainment at individual UK medical schools have shown that graduate-entry students have performed comparably (Manning & Garrud, 2009) or better (Price & Wright, 2010) than undergraduate students in common assessments during the shared full-time clinical phase of those programmes. Some studies (e.g. Bodger et al, 2011) have attempted to identify predictors of attainment in graduate-entry programmes, with mixed conclusions, but commonly that prior academic record (e.g. secondary or tertiary educational qualifications) is a reliable predictor. At present, therefore, there is no evidence about the relative success of graduates who have gone through the graduate-entry vs. the undergraduate medicine courses. There is also very little evidence at a national, pan-individual school level, about markers of success in these different types of course for those students with a prior degree. Two key questions concern the subject of that prior degree and its class or grade. Earlier, secondary educational record may also be an important factor in success. Age, gender, socioeconomic status, and ethnicity may also be relevant factors.

References

  1. Bodger, O., et al. Graduate entry medicine: selection criteria and student performance. PloS one, 6(11), (2011): e27161.
  2. James, D, Chilvers, C. Academic and non‐academic predictors of success on the Nottingham undergraduate medical course 1970–1995."Medical education 35.11 (2001): 1056-1064.
  3. McManus, I. C., et al. "The UKCAT-12 study: educational attainment, aptitude test performance, demographic and socio-economic contextual factors as predictors of first year outcome in a cross-sectional collaborative study of 12 UK medical schools." BMC medicine 11.1 (2013): 244.
  4. Manning, G, Garrud, P. Comparative attainment of 5-year undergraduate and 4-year graduate entry medical students moving into foundation training. BMC medical education 9.1 (2009): 76.
  5. Price, R. Wright, SR. Comparisons of examination performance between 'conventional' and Graduate Entry Programme students; the Newcastle experience. Medical teacher 32.1 (2010): 80-82. • Wilkinson, TJ., et al. Are differences between graduates and undergraduates in a medical course due to age or prior degree?. Medical education 38.11 (2004): 1141-1146.

Professor Chris McManus
i.mcmanus@ucl.ac.uk
UKMEDP003
Do the Educational Performance Measure decile score and SJT predict successful completion of the foundation programme?

Approved on 15 October 2015 UKMED Advisory Board

Published in BMJ Open in July 2018 as "Evaluating the validity of the selection measures used for the UK’s foundation medical training programme: a national cohort study"
Summary: Please outline your proposed research. Work Psychology Group published a report in February 2015 on the Validation of the F1 Selection Tools . The study was limited to 391 F1s and therefore did not have data on which trainees successfully completed the foundation programme. Their sample specifically targeted F1 doctors who had received particularly high or particularly low SJT scores. They make the following recommendation: “that further studies are undertaken to explore the relationship between performance at application and performance outcomes beyond F1 (for example at the end of F2 and into specialty training) and that application scores (particularly SJT scores) spanning the full range of scores are targeted. If the relationship between application scores and ARCP outcomes is to be examined further, a large population (ideally all schools) should be targeted, as incidences of unsatisfactory ARCP outcomes appear to be very rare (1.1% in the present sample).” We will seek to measure the predictive validity of the Educational Performance Measure (EPM) decile score (note in 2012 the EPM score was in quartiles) and the Situation Judgement Test (SJT) , collected as part of medical students’ applications to the foundation programme captured on the Foundation Programme Application System (FPAS) using ARCPs from the two-year foundation programme as an outcome measure. The EPM deciles are a medical school performance score calculated by the applicant’s medical school based on performance in a number of assessments and divided into 10 equal groups (deciles) within the given medical school and not UK-wide. Each UK medical school has agreed with its students which assessments will be included in this measure. We will also seek to understand the relationship between the EPM decile score and SJT scores and specialty recruitment (applications to specialty training made by F2 doctors) . Specifically we will investigate: whether the EPM decile score and SJT relate to which specialties F2 doctors apply to and whether offers of a place on the training programme are made. The output of this research will allow us to fulfil our duties outlined in The Trainee Doctor “10 Periodically, the GMC will analyse evidence from these sources to draw together a picture of the state of foundation and specialty training throughout the UK. This will show performance against standards by postgraduate deaneries, LEPs, medical Royal Colleges and Faculties and specialty associations and will seek to show which factors are most significant in predicting good and poor educational outcomes within training programmes and at the end of training “ It will also assist Medical Schools in meeting the following standard: “172 Quality management will involve the collection and use of information about the progression of students. It will also involve the collection and use of information about the subsequent progression of graduates in relation to the Foundation Programme and postgraduate training, and in respect of any determinations by the GMC.” From Tomorrow’s Doctors (2009) Note that from January 2016, these standards will be superseded by Promoting excellence: standards for medical education and training , the relevant requirement will be “R2.5 Organisations must evaluate information about learners’ performance, progression and outcomes – such as the results of exams and assessments – by collecting, analysing and using data on quality and on equality and diversity” The output of the analysis will help us begin to explore whether the EPM deciles and the SJT score are predictive of trainee performance as measured by ARCP and recruitment outcomes during the foundation programme and therefore whether it is appropriate to split medical school cohorts into groups based on EPM deciles and or SJT to report on their progress at foundation ARCP and at recruitment into specialty training from F2 in order to provide more granular reports on outcomes to medical school deans. We will use the output to enhance the existing medical school progression reports. This could, if the analysis shows that it would be appropriate, allow a medical dean to see the ARCP and recruitment outcomes of their top performing graduates versus their less well performing graduates; where the definition of these groups is determined by this proposed analysis. We may find that ARCP outcomes for foundation are not suitably granular for establishing whether EPM and SJT are useful predictors of which trainees will struggle in foundation training. To establish whether the SJT and EPM predict which trainees required additional support from their foundation schools would require UKMED to have identifiable data on doctors in difficulty on the foundation programme. These may be doctors who received support in order that they were able to achieve an outcome 1 or an outcome 6 and therefore may not have been awarded an unsatisfactory outcome. The UKFPO Annual report for 2014 says there were 186 F1 and 163 F2s from UKMED medical school monitored via foundation schools’ doctors in difficulty (DiD) policies and processes. We do not know which ARCP outcomes these DiD s were finally awarded. The output will also allow comparison of the ARCP and the recruitment outcomes across foundation schools adjusting for the prior attainment of the trainees (i.e. their performance on entry to the foundation programme). We know from the FPAS application handbook that higher ranking applicants get their first choice of school. So viewing outcomes by foundation school with no adjustment may reflect the given school’s intake more than the training it offers. In addition the output will inform our work on differential attainment. We will be able to see whether ethnicity and gender adds to the prediction of ARCP and recruitment outcomes, after accounting for prior attainment as measured by the EPM decile score and SJT in the model. We will seek to understand the variation in EPM decile score and SJT by ethnicity, gender and socio-economic status. Finally we will seek to understand whether the EPM decile score and SJT relate to trainee’s own rating of their level of preparedness as captured on the NTS 2015. This will inform our work on the construct validity of the NTS preparedness indicator. The outputs of this work could be used to help the GMC decide how the SJT and EPM decile scores might contribute to a UK licensing assessment. The analysis might help the UKPFO consider the best way to rank students for entry to the foundation. There are limitations to this study; the UKMED phase 1 cohort does not include graduates from non-UK medical schools who do apply to Foundation training using the FPAS system. In future years UKMED may include those non-UK graduates from the point they apply to foundation training. Note these cases will never have data from the Higher Education Statistics Agency. Any conclusions will therefore be limited to UK graduates. Note that SJT data are only available from 2013 onwards; in 2012 SJT was piloted for the first time and the 2012 parallel recruitment exercise data are not included within UKMED. The 2012 parallel recruitment exercise is described in the Medical School Council report. Those who entered foundation in 2013 will now have a complete set of ARCP outcomes for the two years of their foundation programme so a complete cohort study is possible.

References

  1. Patterson, F., Ashworth, V., Zibarras, L., Coan, P., Kerrin, M. and O'Neill, P. (Sep 2012). Evaluations of situational judgement tests to assess non-academic attributes in selection. Medical Education, 46(9), 850-868. doi: 10.1111/j.1365-2923.2012.04336.x Patterson et al conducted a review of the emerging international research evidence for the use of situational judgement tests (SJTs.) They conclude further research is required to explore theoretical developments and the underlying construct validity of SJTs.
  2. Simon E1, Walsh K, Paterson-Brown F, Cahill D. (Feb 2015). Does a high ranking mean success in the Situational Judgement Test? Clinical Teacher. 2015 Feb;12(1):42-5. Simon et al (2015) report on the relationship between EPM decile scores and SJT scores; but the data were harvested from trainees in a self-reported survey rather than directly from the UKFPO. Their survey achieved a response rate of 8% (N= 3,175 – 12 medical school); so their finding that there is no relationship between the EPM decile score and the SJT may not be correct. It is not clear why this approach was taken rather than obtaining the data directly from the UKFPO.
  3. Patterson F., Lievens F, Kerrin M., Munro N , Irish B. . The predictive validity of selection for entry into postgraduate training in general practice: evidence from three longitudinal studies. British Journal of General Practice 2013 ; 63: 734 – 741 Patterson et al (2013) found that the SJT accounted for 6% of the variation in end of GP training assessments, but is not clear if this finding relates to the AKT or the CSA.
  4. Patterson F, Carr V, Zibarras L, et al (2009). New machine-marked tests for selection into core medical training: Evidence from two validation studies. Clinical Medicine; 9(5): 1–4. Patterson concluded that the SJT was the best single predictor of CMT interview scores, but does not use outcomes from training e.g. MRCP membership exam results to explore the predictive validity of the SJT for CMT trainees. The authors note that The 2008 CMT sample comprised only a subset of applicants – those who applied to both CMT and GP. It could be argued that this sample may not fully represent the CMT applicant population as a whole, so that results may not readily generalise to the applicants not included in this sample.

Mr Daniel Smith
dsmith@gmc-uk.org
UKMEDP020
The role of academic attainment in understanding sex differences in specialty choice and fitness to practise.

Approved on 15 June 2016 UKMED Advisory Board

Published in BMJ Open in March 2019 as "Effect of sex on specialty training application outcomes: a longitudinal administrative data study of UK medical graduates"
The proportions of men and women in different medical specialties varies greatly [1]. Understanding how and why is important for effective workforce planning and the provision of future healthcare, and to reduce sex segregation in some specialties. It may also help us understand other areas of stark sex differences, such as disciplinary action, where male doctors have nearly 2.5 times the odds of facing medico-legal action [2], and doctors from certain specialties are at higher risk of receiving sanctions [3]. Sex differences in specialty choice are partly explained by features such as how plannable, technical, and intellectual a specialty is [1]; but success in obtaining a training place depends on competition ratios, selection methods, and candidates’ previous academic attainment – also potentially associated with sex. Academic performance is also important to help us understand how sanctions relate to sex and specialty, because past academic performance predicts future academic performance [4], and poor academic performance is associated with increased odds of sanctions [5].

References

  1. Elston MA. Women and medicine: the future. London: Royal College of Physicians, 2009.
  2. Unwin E, Woolf K, Wadlow C, et al. Sex differences in medico-legal action against doctors: a systematic review and meta-analysis. BMC Medicine 2015; 13:172.
  3. Unwin E, Woolf K, Wadlow C, et al. Disciplined doctors: does the sex of a doctors matter? A cross-sectional study examining the association between a doctor’s sex and receiving sanctions against their medical registration. BMJ Open 2014; 4:8.
  4. McManus IC, Woolf K, Dacre J, et al. The Academic Backbone: longitudinal continuities in educational achievement from secondary school and medical school to MRCP(UK) and the specialist register in UK medical students and doctors. BMC Medicine 2013; 11:242.
  5. Papadakis MA, Arnold GK, Blank LL, Holmboe ES, Lipner RS. Performance during Internal Medicine Residency Training and Subsequent Disciplinary Action by State Licensing Boards. Ann Intern Med 2008; 148(11):869-876

Doctor Emily Unwin
emily.unwin.12@ucl.ac.uk
UKMEDP022
The allocation of doctors to specialty and general practice training posts by demographic and socio-economic characteristics

Approved on 15 June 2016 UKMED Advisory Board

Published in UKMED in July 2019 as "A sequential analysis of the specialty allocation process in the UK. Empirical evidence from the UKMED database."
Becoming a medical practitioner is a competitive and complex process and its outcome determines the composition of the medical profession. There is a growing concern that the profession should reflect both the appropriate skills and a balance of social, economic, gender and ethnicity (GMC 2010) The distribution of these characteristics is highly unequal both across medical specialties and between specialties and general practice (Goldacre, Laxton et al. 2010 and Rodríguez Santana,& Chalkley 2015). There is a persistent gender gap in certain specialties (men in surgery, women in general practice), and underrepresentation of those from deprived socioeconomic backgrounds (Arumpalam, Naylor et al. 2005) in highly competitive specialties. We propose to analyse the outcome of the specialty allocation, recognising that it is a sequential process. Junior doctors’ preferences over the different training posts and their personal characteristics, qualifications and environment influence their applications; these are then assessed to determine the suitability of applicants to each training post and finally doctors’ decide which of the offers they have received to accept. Our principle objective is to understand how demographic and socio-economic characteristics impact on each stage of this process; how do an individual’s characteristics correspond to their decision to apply, to their subsequent assessment and to their decisions to accept offers. Such an understanding is vital for the formulation effective strategies to ensure greater representativeness across specialities and general practice. Central to our approach is controlling for other factors, such as previous educational experience and attainment to establish the effect of demographic and socio-economic characteristics "other things equal".

References

  1. GMC 2010 This report highlights the importance of ensuring the equality, diversity and opportunity in the specialty and GP recruitment process. One of the GMC’s main goals is to widen access and participation and to ensure that the selection process for entry specialty and GP recruitment are fair, transparent and effective.
  2. Goldacre MJ, Laxton L, Lambert T. Medical graduates’ early career choices of specialty and their eventual specialty destinations: UK prospective cohort studies. BMJ: British Medical Journal. 2010;341 This study compares the specialty choices of graduates over time finding that some specialties consistently attract women (e.g. general practice, paediatrics) or men (e.g. surgical). Moreover it shows clear mismatches between early career choices and eventual destinations, pointing out the importance of medical school, undergraduate and foundation training in the eventual choices.
  3. Rodríguez Santana, I., & Chalkley, M. J. (2015). The socioeconomic and demographic characteristics of United Kingdom junior doctors in training across specialities.(pp. 1-15). (CHE Research Paper; No. 119). York, UK: Centre for Health Economics, University of York. This paper analyses the distribution of socio-economic and demographic characteristics for the doctors in training in the year 2013. The authors find systematic differences between specialties in terms of gender, ethnicity, age, origin and socio-economic background.
  4. Arulampalam W, Naylor R, Smith J. Doctor Who? Who gets admission offers in UK medical schools. IZA Discussion Paper No. 1775. 2005 This study shows that individuals from disadvantaged socioeconomic backgrounds, mature students and ethnic minorities have a lower probability of receiving an offer from medical schools in the United Kingdom. A similar analysis in the specialty allocation process would test if non-majority candidates are prejudiced.
  5. Fang, H. and A. Moro (2010). Theories of statistical discrimination and affirmative action: A survey, National Bureau of Economic Research. This paper summarize several statistical discrimination theories (as opposite to tasted-based theories of discrimination) that derive group inequality without assuming racial, gender or socioeconomic animus, or preference bias, against a target group. These theories can be used to explain some of observed differences in the specialty recruitment outcomes. For instance, certain groups of doctors might perceive themselves as less qualified or non-suitable and apply in a smaller ratio to the specialties with the greatest competition ratios. Alternatively, the lack of role models may act as a disincentive to apply to certain specialties (e.g. women in surgical specialties).

Professor Martin Chalkley
martin.chalkley@york.ac.uk
UKMEDP026
“Getting on” in medicine: a programme of study of careers trajectories and decisions of doctors

Approved on 25 February 2016 UKMED Advisory Board

Published in BMJ Open in September 2017 as "The relationship between school type and academic performance at medical school: a national, multi-cohort study" Published in BMJ Open in June 2018 as "Relationship between sociodemographic factors and selection into UK postgraduate medical training programmes: a national cohort study" Published in BMC Medical Education in December 2018 as "Geographical mobility of UK trainee doctors, from family home to first job: a national cohort study" Published in BMJ Open in March 2019 as "Relationship between sociodemographic factors and specialty destination of UK trainee doctors: a national cohort study"
Millburn's "Fair Access to Professional Careers" highlighted that the increasing diversity of the UK medical student body in terms of age, ethnicity and gender is not reflected in terms of student socio-economic (SEC) background. Cleland et al's (2012) subsequent work identified that research is lacking on how widening access (WA) students and doctors progress in medicine. Only one recent UK study does so. Dowell et al. (2015), in a survey of 2050 Scottish GPs, found that those whose parents had semi-routine or routine occupations were more likely to be working in a deprived practice than those from professional families. While this suggests that SEC on entry to medical school may be associated with differences in career pathway, the study looked only at a sub-sample of doctors who were established in their career choice. In contrast, we are interested in "tomorrow's doctors", the generation who are currently deciding what (specialty) and where (location) they wish to work - or, indeed, when or if they wish to work as a doctor after the Foundation Programme. Are their careers decisions influenced by demographic factors such as SEC only or is there a complex relationship between these and other factors (e.g., medical school) given the medical training pathway is competitive (see MacKenzie et al. in press)? We must understand these relationships to identify how to address barriers to progression within medicine and to inform policy decisions about workforce planning. WA students are few and only UKMED provides sufficient data to address these questions appropriately.

References

  1. Cleland JA, Dowell J, McLaughlin J, Nicholson S, Patterson F. (2012) Identifying best practice in the selection of medical students. This GMC-commissionned literature review collated data on WA activities directed at preparing potential applicants, the application process, support once at medical school, and what happens to WA medical students when they become doctors. The relevance to this study is that they identified that data is lacking in terms of understanding the career pathways of students from WA background.
  2. Cleland JA, Johnston P, Watson V, Krucien N, Skatun D. (due for publication in Medical Education Feb 2016 issue). What do UK doctors-in-training value in a post? A discrete choice experiment. This study, which used a novel methodology to progress medical careers decision making from information-seeking survey, to identify the relative strength, or value, of careers preferences, found that trainees placed most value on good working conditions, good opportunities for their partner and desirable geographical location when making careers-related decisions. (See below for relevance).
  3. Dowell J, Norbury M, Steven K, Guthrie B. (2015) Widening access to medicine may improve general practitioner recruitment in deprived and rural communities: survey of GP origins and current place of work. BMC Medical Education 2015, 15:165 doi:10.1186/s12909-015-0445-8. Relevance: the first study to indicate that the careers pathways of WA and traditional doctors may differ, and hence reinforce the need for a large-scale study in this area.
  4. MacKenzie RK, Cleland JA, Ayansina D, Nicholson S. (In submission). Does the UKCAT predict performance on exit from medical school? A national cohort study. This project, funded by the UKCAT, was the first to link the UKCAT and UKFPO databases, to examine predictors of performance during, and on exit from, medical school. They found that those from lower IMD groups perform less well on the UKFPO selection processes. Relevance: disadvantage may continue given the competitiveness of medical education and the UK medical training pathway. There is a need to explore the relationship between SEC, other demographics, performance and careers decisions/trajectory.
  5. Svirko E, Goldacre MJ, Lambert T. Career choices of the United Kingdom medical graduates of 2005, 2008 and 2009: Questionnaire surveys. Medical Teacher. 2015; 355, 365-375. This study indicated that specialty preferences expressed by newly qualified doctors, notably the shortfall in numbers choosing general practice, remain inconsistent with future service needs. Relevance: The findings of the Svirko et al. and Cleland et al. papers are supported by training recruitment and retention figures over recent years and the crisis in workforce planning is increasing (e.g., 50%+ of this year's FY2 cohort stating that they have not applied to go directly into core/specialty/GP training). It is critical to understand the factors influencing careers decision making to plan ways to anticipate, and address, likely mismatches between the careers preferences of newly qualified doctors and healthcare delivery requirements (the right people with the right skills, in the right place, at the right time).

Professor Jennifer Cleland
jen.cleland@abdn.ac.uk
UKMEDP030
What demographic and educational factors predict doctors' decisions to apply for training programmes in particular medical specialties?

Approved on 15 June 2016 UKMED Advisory Board

Published in BMC Medicine in December 2017 as "Factors associated with junior doctors’decisions to apply for general practice training programmes in the UK: secondary analysis of data from the UKMED project" Published in BJPsych Bulletin in May 2019 as "Sociodemographic and educational characteristics of doctors applying for psychiatry training in the UK: Secondary analysis of data from the UK Medical Education Database project."
Decisions to apply for postgraduate training programmes in particular medical specialties are shaped by a multitude of factors. These include the doctor's family background, schooling, undergraduate (medical school) training, early postgraduate professional experience, and the perceived likelihood of a successful application. Much survey-based research has been done into factors that are associated with the self-reported (intended) career preferences of both medical students and early-career doctors [1-3]. Intentions to follow a particular career pathway however are subject to change [4] and may not, at the ultimate decision point, materialise as actual applications to particular specialty training programmes. The UKMED database provides a unique and valuable opportunity to add to the literature on career choice by permitting analysis, for a large cohort of doctors and a range of nationally-recruited UK specialty training programmes, of some of the important factors that are associated with the decision to apply for a place on those programmes. Research in this area has particular importance for workforce planning. The most recent data on application ratios suggests that certain specialties are experiencing recruitment difficulties; from 2013 to 2015 General Practice, Paediatrics and Psychiatry consistently ranked within the five least competitive specialisms [5]. This proposed research may therefore influence changes to policy and practice in areas such as the provision of specialty training programmes, the design of undergraduate medical school courses, methods of recruitment and selection to medical school and the widening participation agenda, in order to encourage a better match between graduate career choice and service need.

References

  1. Querido SJ, Vergouw D, Wigersma L, Batenburg RS, De Rond MEJ & Ten Cate OTJ (2016) Dynamics of career choice among students in undergraduate medical courses. A BEME systematic review: BEME guide no. 33. Medical Teacher 38(1), 18-29. As a systematic review this paper identifies studies relevant to the field, assesses their methodological strength, identifies a wide range of factors associated with career choice, highlights the gaps in the literature and in particular points to the prevalence of studies that analyse expressed student preferences (collected through questionnaires) rather than actual applications.
  2. Svirko E, Goldacre MJ & Lambert T (2013) Career choices of the United Kingdom medical graduates of 2005, 2008 and 2009: Questionnaire surveys. Medical Teacher 35(5), 365-375. By comparing the expressed preferences of three different cohorts of graduates over a four-year period, this paper uncovers the direction of changes in student preferences and highlights an increasing trend towards uncertainty of achieving a post in students’ preferred specialty as a major factor in career choice as well as pointing to the inconsistency between career preference and service need.
  3. Wiener-Ogilvie S, Begg D & Dixon G (2015) Foundation doctors career choice and factors influencing career choice. Education for Primary Care 26(6), 395-403. This study in Scotland uncovers some of the reasons given by medical students for career choices, highlighting, for example, the importance of the medical schools themselves in influencing career decisions.
  4. Lambert TW, Davidson JM, Evans J & Goldacre MJ (2003) Doctors' reasons for rejecting initial choices of specialties as long-term careers. Medical Education 37(4), 312-318. Recognising as a starting point that initial expressed preferences for particular specialties frequently do not match ultimate career paths, this study uses questionnaires to uncover some of the reasons given for rejecting initial choices (e.g. quality of life issues) and examines the relative importance of not considering a particular specialty in the first place versus subsequent rejection of a specialty choice as explanations for application shortfalls in certain specialties.
  5. The NHS, Specialty Recruitment: Competition Ratios. This website provides data on the competition for specialty placements through comparison of applications to specialty training against the number of places available for that specialism, allowing identification of application shortfall in specific areas, i.e. Psychiatry, General Practice and Paediatrics.

Doctor Tom Gale
thomas.gale@plymouth.ac.uk
UKMEDP031
Prediction of performance in the Kent, Surrey and Sussex Foundation Year 1 Regional Prescribing Assessment from the national undergraduate Prescribing Safety Assessment results

Approved on 29 June 2020 UKMED Advisory Board

Background - The Prescribing Safety Assessment (PSA) has been developed as a reliable and validated tool designed to demonstrate medical students’ competence in relation to the safe and effective use of medicines. This is often followed by individual NHS Trusts safe prescribing assessments at the start of the Foundation Year 1 (F1). The interrelationships between assessments of safe prescribing competence in the medical education continuum merits investigation as this will enable the optimisation of the assessment process. Aim - To determine whether there is a correlation between scores in the PSA and the Health Education Kent, Surrey and Sussex (HEKSS) regional prescribing assessment (RPA) for F1 doctors. In addition, agreement between the two tests in pass/fail decisions will be investigated. Methods - A retrospective data linkage study in which individual scores from the 2014 - 2016 PSAs will be identified and matched to scores from the RPA for F1 doctors in the Kent, Surrey and Sussex region over the same period. Thereafter the data will be pseudonymised. Correlation coefficients will be calculated to assess the relationship, if any, between candidates’ scores in the PSA and RPA. In addition, the Kappa coefficients will be used to determine agreement between the two tests in pass/fail decisions. Expected value of results - The results might validate the HEKSS RPA as a robust assessment of safe prescribing competence for F1 doctors which might be applicable nationally. Furthermore, this analysis might enable the prediction of candidates who require additional training in prescribing during their F1.

References

  1. Dornan T, Ashcroft D, Heathfield H, Lewis P, Miles J, Taylor D, Tully M, Wass V. An in depth investigation into causes of prescribing errors by foundation trainees in relation to their medical education - EQUIP study. GMC commissioned study 2009. This study highlights that Foundation Year 1 and Year 2 doctors, who are responsoble for majority of the prescribing in UK hospitals, have the highest error rate. Furthermore, the main targets for interventions to minimise prescribing errors are centred around education.
  2. Heaton A, Webb DJ, Maxwell SRJ. Undergraduate preparation for prescribing: the views of 2413 UK medical students and recent graduates. Br J Clin Pharmacol 2008; 66:128-134 3. Rothwell C, Burford B, Morrison J, Morrow G, Allen M, Davies C, Baldauf B, Spencer J, Johnson N, Peile E, Illing J. Junior doctors prescribing: enhancing their learning in practice. Br J Clin Pharmacol 2012; 73:194-202 References 2 and 3 highlight that Medical students do not feel adequately prepared for prescribing in the foundation years after graduation.

Doctor Michael Okorie
m.okorie@bsms.ac.uk
UKMEDP032
What factors predict doctors' successful completion of core training in medicine and anaesthetics and their subsequent decisions to pursue higher specialty training

Approved on 27 March 2017 UKMED Advisory Board

Core training programmes for medicine and anaesthesia have high fill rates for CT1 entry compared to other specialties. However, these specialties suffer from below optimal conversion rates between core and higher specialty training posts.[1] As a result, there are many unfilled posts at entry to higher specialty training in medicine and anaesthetics. The CfWI has identified an urgent need to increase the number of ST3 posts, and to model the way in which the output from core training posts and ACCS flows into higher specialty training.[2,3] Research is required to understand the extent of attrition between core training and specialty training posts in these specialties, and factors that predict successful appointment to higher specialty training. The main aim of our study is to identify factors, which predict doctors' successful completion of core training in medicine and anaesthesia, and their subsequent decisions to pursue higher specialty training. Previous work has investigated trainees’ perceptions regarding the weighting of individual and job-related factors influencing choice and selection to specialty training posts, but there is limited longitudinal research investigating factors which predict successful completion of training.[4] A large longitudinal prospective study, identified that previous academic attainment predicts undergraduate attainment in pre-clinical and clinical years of a medical degree, but socio-demographic factors are also important predictors of future clinical performance.[5] The UKMED database provides a unique opportunity to investigate the contribution of a number of factors that predict successful completion of core training in medicine and anaesthesia, and successful progression to higher specialty training.

References

  1. Royal College of Anaesthetists (2016). Workforce data pack. Accessed online, 29th Jan 2017. Provides accurate and up to date anaesthetic workforce data collated from College Census 2015, the Centre for Workforce Intelligence’s review of anaesthetics and intensive care medicine, and National Recruitment Office data.
  2. Centre for Workforce Intelligence (2015). In depth review of the acute medical care workforce. Accessed online, 29th Jan 2017. Comprehensive review of all fully trained physicians who contribute to acute medical care including, acute medicine specialists, geriatricians, and physicians from a number of other specialties, with projected analyses of the balance between patient/service demand and supply of CCT holders until 2033.
  3. Centre for Workforce Intelligence (2015). In depth review of the anaesthetics and intensive care medicine workforce. Accessed online, 29th Jan 2017. Comprehensive review of all fully trained physicians who contribute to anaesthetics and intensive care medicine, with projected analyses between patient/service demand and supply of CCT holders until 2033.
  4. Patterson F, Knight A, Dowell J, Nicholson S, Cousans F, Cleland J. (2016) How effective are selection methods in medical education? A systematic review. Medical Education 50(1):36-60. Large systematic review assessing effectiveness and fairness of different selection methodologies for medical training, and highlighting lack of predictive validity studies investigating successful completion of training programmes and progression.
  5. Stegers-Jager KM, Themmen APN, Cohen-Schotanus J and Steyerberg EW. (2015) Predicting performance: relative importance of students’ background and past performance. Medical Education 49(9): 933–45. Large longitudinal prospective study, where multivariate logistic regression analysis identified that previous academic attainment predicts undergraduate attainment in pre-clinical and clinical years of a medical degree, but socio-demographic factors are also important predictors of future clinical performance.

Doctor Tom Gale
thomas.gale@plymouth.ac.uk
UKMEDP038
How do the professional outcomes of medical graduates from gateway courses compare to graduates from standard entry medicine courses?

Approved on 28 March 2017 UKMED Advisory Board

Published in BMC Medical Education in January 2020 as "A comparison of undergraduate outcomes for students from gateway courses and standard entry medicine courses" Published in BMC Medical Education in May 2023 as "How do the post-graduation outcomes of students from gateway courses compare to those from standard entry medicine courses at the same medical schools?"
The postgraduate outcomes of those on 6-year medicine with a gateway courses will be compared to those on the 5 year standard entry courses after accounting for attainment and aptitude measured on entry and the students’ performance relative to their year within the school on exit, used to rank applicants to the foundation programme. UKMED Data are available for three schools that offered both types of course. Outcomes will be measured in terms of: progression through training as captured by ARCPs, performance in royal college medical exams and the specialty training programmes applied to and offered at CT1/ST1.

References

  1. There are two published papers comparing results on the 6-year programme at Kings to the 5-year programme and one paper concerned with results at Southampton. Garlick PB, Brown G: Widening participation in medicine. British Medical Journal 2008, 336:1111–1113. Garlick and Brown report on the entry criteria and the within school performance of the students on the 6-year programme and compare them to those on the 5-year programme. They report that the students on the two programmes have identical first time pass rates during the clinical years of the programme. They conclude that medical students admitted with lower grades can succeed if the grades were obtained at a low achieving school.
  2. Mahesan N, Crichton S, Sewell H, Howell S: The effect of an intercalated BSc on subsequent academic performance. BMC Med Educ 2011, 11:76. Mahesan et al report that students on the 6-year programme at King’s attained Year 5 results which were on average over 4 points lower than students on the regular 5-year programme (95% CI -6.30 to -2.21).
  3. Curtis SA et al. Successful widening access to medicine. Part 2: Curriculum design and student progression. The Royal Society of Medicine 2014; 107 (9):393-397 Curtis et al report that 85% of students on the 6 year programme at Southampton who progress to year 1, graduate with a medical degree in comparison to 95% of students from the 5 year programme There are no published papers comparing the progression of those on the 6-year to the 5-year course at Norwich.
  4. There are two large multicentre UKCAT studies looking at the relationships between performance at medical school and demographic variables, attainment on entry and aptitude on entry: Predictive validity of the UKCAT for medical school undergraduate performance: a national prospective cohort study BMC Medicine 2016 14:140, Paul A. Tiffin, Lazaro M. Mwandigha, Lewis W. Paton, H. Hesselgreaves, John C. McLachlan, Gabrielle M. Finn and Adetayo S. Kas The UKCAT-12 study: Educational attainment, aptitude test performance, demographic and socio economic contextual factors as predictors of first year outcome in a cross-sectional collaborative study of twelve UK medical schools BMC Medicine 2013, 11:244
  5. IC McManus, Chris Dewberry, Sandra Nicholson and Jonathan S Dowell. Although the schools are not named in either of these studies, they cannot have included Kings or Southampton as neither school is listed in the table of schools providing medical school progression data to the UKCAT consortium . Norwich will have been included.

Mr Daniel Smith
Daniel.smith@gmc-uk.org
UKMEDP039
What factors lead to success in obtaining an ophthalmology specialty training (OST) post and completing postgraduate ophthalmology examinations?

Approved on 12 June 2017 UKMED Advisory Board

Published in BMJ Open Ophthalmology in July 2021 as "Predictors of ophthalmology career success (POCS) study"
Run-through surgical specialty training posts lead directly to a certificate of completion of training, making them attractive, but competitive career choices among doctors (1). Ophthalmology specialty training (OST) is a highly subscribed run-through post with competition ratios higher than the overall average for all specialty-training posts (2). We aim to explore which factors lead to successful applications to ophthalmology training on the first attempt. We also aim to understand what influences success in ophthalmology postgraduate examinations on the first attempt. The Royal College of Ophthalmologists’ (RCOphth) endorses this study. The RCOphth oversees a rigorous recruitment process but has little evidence to guide them on the predictive validity of their recruitment measures (personal communication). Understanding what makes their candidates successful in gaining a training post is a research priority. We aim to ascertain whether interview and shortlisting scores predict exam performance and recruitment into ophthalmology specialist training on the first attempt. Our study explores a range of demographic, socioeconomic (4) and academic variables, which are hypothesised to influence success in attaining an Ophthalmology training post, as well as success in passing post-graduate exams on the first attempt. Previous studies have demonstrated the predictive validity of prior educational attainment (PEA) and UKCAT scores in predicting success in medicine (5). Our project aims are twofold; 1) To enable RCOphth and other postgraduate colleges to operate an evidence-based recruitment process and 2) To guide applicants of the factors that may influence success in ophthalmology so that they are able to make realistic career choices.

References

  1. Carr A, Marvell J, Collins J., (2013) Applying to specialty training: considering the competition. BMJ Careers. [Accessed 29/01/2017]. The authors of this paper examine the competition data for all UK specialty training posts in 2013. The purpose was to inform applicants of the relative numbers of posts available in each specialty so that candidates are able to make realistic career choices.
  2. Kennedy C., (2015) Specialty training applications for entry in 2016: competition ratios and the application process. BMJ Careers. [Accessed 29/01/2017]. This paper examines the overall competition ratios for entry into specialist training programmes over a one-year period, with the aim of facilitating the application process for candidates.
  3. McNally SA., (2008) Competition ratios for different specialties and the effect of gender and immigration status. JR Soc Med;101 (10):489-492 This paper examines competition ratios for postgraduate specialties and the likely success rates of candidates. In particular, the author looks at whether or not gender and immigration status are associated with higher success rates in attaining specialty-training posts.
  4. Rodriguez-Santana I, Chalkley MJ., (2015) The socioeconomic and demographic characteristics or United Kingdom junior doctors in training across specialities. CHE Research Paper; Mo 119. York, UK: Centre for Health Economics, University of York. [Accessed 29/01/2017]. This paper draws upon the National Training Survey data to analyse the differences in socioeconomic and demographic characteristics of doctors in all postgraduate specialties in 2013.
  5. I. C. McManus, C. Dewberry, S. Nicholson, and J. Dowell (2013) The UKCAT-12 study: Educational attainment, aptitude test performance, demographic and socio-economic contextual factors as predictors of first year outcome in a collaborative study of twelve UK medical schools. BMC medicine 11.1: 244 The authors of this study present their findings from a prospective study of over 5000 medical students in twelve UK medical schools. The study examines the predictive validity of multiple academic variables including prior educational attainment (PEA) and UKCAT scores in predicting medical school performance.

Doctor Aditi Das
aditi.das@moorfields.nhs.uk
UKMEDP041
Development of a UKMED multidimensional measure of widening access status.

Approved on 25 September 2017 UKMED Advisory Board

Published in UKMED in March 2021 as "Development of a UKMED multidimensional measure of widening participation status. "
Contextual admissions data are increasingly employed in selection to the study of medicine in the UK, albeit with little knowledge about the quality of the indicators or the implications of their use on both widening access (WA) and student achievement. [1,2,3] Moreover, there is concern about the validity of contextualised admissions decision making because; contextual data have been shown to produce conflicting information on WA status, doubtful veracity of self-reported information and the extent of missing data values on contextual indicators generally [4] [5] Furthermore, little is known about the association between students’ contextual background characteristics and performance in medical school.[6, 7, 8] Triangulation of contextual indicators to identify those most likely to be disadvantaged is recommended to reduce numbers of false positive WA status identification but, may also serve to increase the number of false negative and introduce new injustices. Contextual data comprise disparate measures of disadvantage which each capture an aspect of the underlying concept WA status. It is therefore important to know about the strengths and limitations of the most commonly used contextual indicators, singly and in combination. It is also desirable to efficiently combine the most reliable contextual indicators into a single multidimensional measure of WA status which UK medical schools can confidently use in their selection processes. The UK Medical Education Database (UKMED) includes a range of contextual admissions indicators commonly used in selection to the study of medicine and provides a unique opportunity to achieve this study’s aims, the outcome of which has the potential to make WA to medicine fair, transparent and above all, evidence-based.

References

  1. Medical Schools Council. Entry requirements for UK medical schools - 2017 entry. MSC 2016.
  2. Garrud P. Help and hindrance in widening participation - commissioned research report. Medical Schools Council - Selecting for Excellence, 2015.
  3. Boliver V., Gorard S., Siddiqui N., Will the use of Contextual Indicators Make UK Higher Education Admissions Fairer? Educational Sciences. 2015; 5(4):306-22.
  4. Steven K., Dowell J., Jackson C., Guthrie B. Fair access to medicine? Retrospective analysis of UK medical schools application data 2009-2012 using three measures of socioeconomic status. BMC Med Educ. 2016;16(1):11.
  5. Tiffin P., McLachlan J., Webster L., Nicholson S. Comparison of the sensitivity of the UKCAT and A Levels to sociodemographic characteristics - a national study. BMC Med Educ[Internet].2014;14(7).

Doctor Paul Lambe
paul.lambe@plymouth.ac.uk
UKMEDP042
Understanding variation in BME medical exam performance across the UK

Approved on 26 September 2017 UKMED Advisory Board

The landscape of exam result differential performance for Black, Asian and Minority Ethnic (BAME) individuals in the medical education system is complex. Performance data from undergraduate and postgraduate settings demonstrate lower success rates in doctors of BAME background. The recent “Fair Training for All” report by the General Medical Council used a qualitative approach to understand the barriers and facilitators to success in BAME doctors. One of the key findings of the report was the negative impact of poor performance in exams: poorer performance in exams adversely affected autonomy in job choice, increased likelihood of being separated from family and support networks, and increased chance of mental health problems. Failing exams can lower confidence, and resits can be felt to interfere with workplace learning. There is relatively little published evidence on the BAME attainment gap, especially when considering local variation. Learning from the NHS clinical arena, identifying regional and local variation in performance is valuable as it (1) highlights exemplars of best practice and (2) helps allocate effort and resource to areas with poorer clinical outcomes. Measuring and publishing unit level NHS performance data is the central tenet of the Health Care Quality Commission. Our planned analysis will use the NHS approach, and aim to identify medical school variation in BAME exam success, identifying where the attainment gap becomes more pronounced and may warrant further explanatory investigation. Central to this project will be to implement appropriate statistical methods to ensure accurate definition of medical school variation. In particular, the concept of ‘case-mix’ from clinical practice is relevant; understanding the characteristics of individuals within each medical school cohort prior to commencing study (e.g. prior academic attainment, socio-demographic background) is essential to interpreting any inter-school variance in attainment gap.

References

  1. Ferguson, E., D. James, and L. Madeley, Factors associated with success in medical school: systematic review of the literature. BMJ, 2002. 324(7343): p. 952-7. [Systematic review providing relevant background on BAME performance in medical school]
  2. Haq, I., et al., Effect of ethnicity and gender on performance in undergraduate medical examinations. Med Educ, 2005. 39(11): p. 1126-8. [Jane Dacre paper - observational cohort study of year 3 undergraduates, looking at relationships between BAME status and performance]
  3. McManus, I.C., et al., The UKCAT-12 study: educational attainment, aptitude test performance, demographic and socio-economic contextual factors as predictors of first year outcome in a cross-sectional collaborative study of 12 UK medical schools. BMC Med, 2013. 11: p. 244. [Sentinal paper looking at pre-medical school performance as a predictor - relevant to our plans for case-mix adjusted analyses]
  4. Woolf, K., et al., Exploring the underperformance of male and minority ethnic medical students in first year clinical examinations. Adv Health Sci Educ Theory Pract, 2008. 13(5): p. 607-16. [Impact of underperformance in BAME cohorts evaluated]
  5. Woolf K, R.A., Viney R, Rigby M, Needleman S, Griffin A., Fair training pathways for all: Understanding experiences of progression. General Medical Council Report, 2016. [Essential GMC report highlighting variation in attainment]

Doctor James Galloway
james.galloway@kcl.ac.uk
UKMEDP043
Does medical school entry performance and medical school performance predict success on the Intercollegiate Membership of The Royal College of Surgeons (MRCS) exam ?

Approved on 25 September 2017 UKMED Advisory Board

Published in Postgraduate Medical Journal in March 2021 as "Performance at medical school selection correlates with success in Part A of the intercollegiate Membership of the Royal College of Surgeons (MRCS) examination" Published in Journal of the Royal Society of Medicine in July 2021 as "The impact of disability on performance in a high-stakes postgraduate surgical examination: a retrospective cohort study." Published in BMJ Open in July 2021 as "Does performance at medical school predict success at the Intercollegiate Membership of the Royal College of Surgeons (MRCS) examination? A retrospective cohort study." Published in Journal of the Royal Society of Medicine in February 2022 as "Differential attainment at MRCS according to gender, ethnicity, age and socioeconomic factors: a retrospective cohort study" Published in BMJ Open in January 2022 as "Does performance at the intercollegiate Membership of the Royal Colleges of Surgeons (MRCS) examination vary according to UK medical school and course type? A retrospective cohort study."
In 2008, McManus and colleagues published an article in BMC Medicine highlighting the substantial difference in performance on the MRCP across medical schools and encouraged other groups to investigate whether similar patterns exist in other postgraduate UK examinations. This study reported data from those medical graduates taking PACES between 1989 and 2005. These cohorts entered medical school in the years before school examination grade inflation and before the widespread implementation of multi-method selection into medical school (e.g., the use of selection tests such as UKCAT, BMAT or GAMSAT). Moreover, the McManus study was also prior to changes in assessment during medical school, notably the introduction of ranking using the educational performance measurement (EPM), which is then used in selection into the next stage of medical training, the UK Foundation Programme. To the best of our knowledge, no previous studies have looked at the relationship between performance on medical school entry tests and performance on postgraduate examinations although UKCAT has been shown to be predictive of medical school final outcome (EPM). Nor has there been prior research looking at performance at medical school and performance on postgraduate examinations. The development of the UKMED database enables such studies. Our specific interest is surgical selection and training. The Intercollegiate Membership of The Royal College of Surgeons (MRCS) is a mandatory postgraduate exam for all aspiring UK surgeons wishing to apply for higher surgical training (ST3 selection). It is attempted by upwards of 6000 UK and overseas doctors annually. We wish to investigate whether medical school, medical school entry performance and medical school performance predict MRCS success.

References

  1. McManus IC, Elder AT, de Champlain A, Dacre JE, Mollon J, Chis L. Graduates of different medical schools show substantial differences in performance on MRCP(UK_ Part 1, Part 2 and PACES examinations. BMC Med 2008; 6: 5. The difference in MRCP performance was explained by candidate's medical school and highlighted the need for others to assess UK medical school performance and all postgraduate examinations. UKCAT, GAMSAT and BMAT were not in place at the time of this study (indeed graduate entry programmes did not exist for the cohorts in McManus et al.'s study but they now make up 10% of medical school places. To date, no other research group has investigated the relationship between medical school, UKCAT/BMAT/GAMSAT performance, medical school performance and performance on UK postgraduate surgical assessments.
  2. Tiffin PA, Mwandigha LM, Paton LW, Hesselgreaves H, McLachlan JC, Finn GM, Kasim AS. Predictive validity of the UKCAT for medical school undergraduate performance: a national prospective cohort study. BMC Medicine 2016; 14: 140. This study demonstrated that UKCAT scores are predictive of undergraduate medical school performance, supporting the concept of the predictive validity of UKCAT.
  3. MacKenzie RK, Cleland JA, Ayansina D, Nicholson S. Does the UKCAT predict performance on exit from medical school? A national cohort study. BMJ open 2016; 6: e011313. This study was the first to link the UKCAT and UKFPO databases and found that UKCAT score is predictive of medical school final outcome. The relationship between UKCAT/BMAT/GAMSAT score, EPM deciles, and mandatory postgraduate UK College examinations has yet to be explored.

Professor Jennifer Cleland
jen.cleland@abdn.ac.uk
UKMEDP044
From family-home to education and from education to training: the spatial patterns of future doctors.

Approved on 25 September 2017 UKMED Advisory Board

Published

Published in BMJ Open in February 2024 as "Determining the distance patterns in the movements of future doctors in UK between 2002 and 2015: a retrospective cohort study"
This project aims to discover and analyse the spatial patterns in the movements of the future doctors: from their family-home to University and from University to training locations. This research is new in two aspects: we propose a robust and spatio-temporal quantitative statistical framework, while the majority of previous research is mainly qualitative; we take into account the starting point of each student pathway (for most of them their family-home postcode, although this may not be true for older entrants) and their geographic trajectories (movements to different post-codes representing education, training and work). This project will focus on the following analysis: 1. Movements from ‘home’ (prior to entering medical studies) through to working as a consultant/GP. The end point will only be possible for those who have obtained their Certificate of Completion of Training (CCT). The majority of these will be GPs as GP training is shorter than the training time for other specialties. (It would be possible to extend the data analysis over time as more trainees within the data base complete a CCT) Currently we have 6,832 doctors in the HESA data now working as GPs If additional resources are available (e.g. more MSc students interested or by applying for funding a PhD), the following analyses (ordered for importance) will be carried out: 2. Home to medical school- full cohort. HESA data include all cases starting at a UK Medical school from 2002 to 2015. 3. Medical school to foundation – only those who have graduated by 2016 so those starting in 2011 and earlier. 4. Foundation to specialty training only those who entered foundation from 2012 (the first year UKFPO foundation application data are available). The data available for each of these analyses will overlap but may not consist of precisely the same cases, therefore they may require different individual analysis. This will represent a significant improvement compared to previous research based on qualitative methodologies (interviews), which focused on a restricted temporal window and did not account fully for the original residence of the student (often the family home).

References

  1. Goldacre et al. 2013. Geographical movement of doctors from education to training and eventual career post: UK cohort studies. J R Soc Med 106: 96-104. This paper has a similar objective of our project, however they used qualitative methods with which it is not possible to estimate the significance and strength of geographical patterns. In addition, the authors focused on historical data (1974 to 2008) which largely precede the current pathways for training (Foundation training from 2005 and revised Specialty training pathways from 2007).
  2. Brice and Corrigan, 2010. The changing landscape of medical education in the UK. Medical Teacher, 32: 727-732. This paper gives a background of the context in which students and doctors’ movements happen. We will take into account in our analysis the changing landscape of medical education.
  3. Goddard et al. 2010. Where did all the GPs go? Increasing supply and geographical equity in England and Scotland. Journal of Health Services Research & Policy 15: 28-35. This work analysed the factors associated with the distribution of GPs across England and Scotland. They do not take into account their education and training pathways. However, their results may help the understanding of the geographical patterns estimated by our project, and give a direction for future analyses.
  4. Parkhouse and Lambert, 1997. Home, training and work: mobility of British doctors. Medical Education 31: 399-407. This paper has a similar objective of our project but it is restricted to 1974 to 1993 only. Again as in Goldacre (2013), they use qualitative methods.
  5. Dowell et al. 2015. Widening access to medicine may improve general practitioner recruitment in deprived and rural communities: survey of GP origins and current place of work. BMC Med Educ. 2015 Oct 1;15:165. doi: 10.1186/s12909-015-0445-8. It is not clear how transferrable to the UK context these findings may be and the benefits to care from socioeconomic diversity amongst clinicians in the UK or other countries is as yet unproven. Rural origin is the factor most strongly associated with subsequent rural practice with evidence from remote areas of the western world that students recruited from rural backgrounds are more likely to practice in under-served remote and rural areas.

L Sedda
l.sedda@lancaster.ac.uk
UKMEDP046
Are there differences between those doctors who apply for a training post in FY2 and those who take time out of training?

Approved on 25 September 2017 UKMED Advisory Board

Published in BMJ Open in November 2019 as "Are there differences between those doctors who apply for a training post in Foundation Year 2 and those who take time out of the training pathway? A UK multicohort study"
Accurately predicting medical workforce supply is increasingly challenging as doctors no longer behave in time-recognised ways in terms of career decision making (Arthur et al., 2005; Cleland et al., 2016, 2017). For example, in the UK context, medical graduates are choosing not to progress through training as predicted. In 2016, nearly 50% of those graduates completing the Foundation Programme did not apply for specialty/GP training at the expected point in time (UKFPO Career Destination Reports 2015, 2016; Scanlan et al., 2017). Simply put, one in two of today’s medical graduates left the training pipeline at the first opportunity to do so in order while keeping their options open (i.e. with full registration and eligibility to apply for higher training). Instead, they opted to take a break from training. The percentage working overseas for a period of time has remained static, the number taking other types of NHS service posts including Development Fellow posts has increased as has those who decided to leave clinical practice. Given this “brain drain”, more understanding of the differences and similarities between those doctors who progress directly from FY2 into training, and those who take time out of training is crucial as this can inform policy and practice in relation to medical selection and attracting trainees to medical training across the breadth of specialties, and thus ensure sufficient doctors to deliver service now and in the future (Collins & Young, 2000; Gorman, 2017).

References

  1. Cleland JA, Johnston P, Watson V, Krucien N, Skatun D. 1) What do UK doctors-in-training value in a post? A discrete choice experiment. Medical Education 2016: 50; 189-202. 2) What do UK medical students value most in their career? A discrete choice experiment. Medical Education 2017: 51; 839-851, and 3) Scanlan et al. Location and Support are Critical to Attracting Junior Doctors: A Discrete Choice Experiment. In submission to Medical Education. This series of studies used a novel methodology to progress understanding of medical careers decision making from data collected in simple, information-seeking surveys, to using an approach which identified the relative strength, or value, of careers preferences, found that trainees and placed most value on good working conditions and location(being near family and friends) when making careers-related decisions. Interesting, female students place more value on location than do male students, which is of relevance given the greater number of female medical graduates nowadays. The third paper is this series focused specifically on FY2s (2016 cohort). It identified that FY2 doctors who applied for a training post placed less value on supportive culture and excellent working conditions than those who did not apply (ie those who were planning to take time out of training). It also identified that male F2s valued geographical locality and a supportive culture less than their female counterparts, and those who entered medicine as graduates placed less value on a desirable location and supportive culture than those who entered medical school as school leavers. The relevance of these papers are that they provide insight into the factors which are important to contemporary cohorts of UK medical students and doctors in training in their careers decision making. However, their focus was generic “push-pull” factors rather than specialty choice (e.g., a preference for surgery or general practice) and they did not investigate possible links between these preferences and specialty preferences.
  2. Gorman D. Matching the production of doctors with national need. Medical Education 2017, early online view. This is a literature-based analysis of the complexity of health system planning and the consequent alignment of medical school selection processes. it concludes that some student selection processes result in desirable (to the planners) career and career location updates. However, the article makes clear that many different factors need to be aligned to lead to change - it is not just admissions but also pedagogy, government investment and integrated medical school and postgraduate training) ie workforce planning is complex so "simple" solutions are inappropriate.

Mr Benard Kumwenda
r0bk15@abdn.ac.uk
UKMEDP051
A comparison of the properties of BMAT, GAMSAT and UKCAT

Approved on 25 September 2017 UKMED Advisory Board

Published in BMJ Open in February 2022 as "Can achievement at medical admission tests predict future performance in postgraduate clinical assessments? A UK-based national cohort study."
Currently the UK Clinical Aptitude Test (UKCAT) and the Biomedical Admissions Test (BMAT) are the two main aptitude tests used for selection into standard medical and dental undergraduate courses in the UK. Previously, both tests have been shown to significantly predict undergraduate performance in medical students (1, 2). However, to date, no direct comparison has been made between the two assessments, in terms of their ability to predict important outcomes, their sensitivity to sociodemographic variables and the degree to which they incrementally add value within the selection system, above and beyond that provided by conventional measures of academic attainment. The Graduate Medical School Admissions Test (GAMSAT) is used for selection into some graduate entry medical courses. There is some weak evidence for the predictive validity of the GAMSAT in the early years of medical school (3, 4). However, no large studies of the predictive validity of GAMSAT in the UK have been conducted. It has also been noted (4) that the GAMSAT appears to possess more predictive performance than the UKCAT when predicting Educational Performance Measure (EPM), but as this was not the primary research aim, this issue was not explored in depth. Comparisons of the properties of the three tests would allow the strengths and relative weaknesses of each of the tests to be evaluated, as well as their potential to widen (or narrow) participation in medicine. Thus, the results will provide selectors with an informed choice.

References

  1. McManus I, Ferguson E, Wakeford R, Powis D, James D. Predictive validity of the BioMedical Admissions Test (BMAT): an evaluation and case study. Med Teach. 2011;33:53 - 7.
  2. Tiffin PA, Mwandigha LM, Paton LW, Hesselgreaves H, McLachlan JC, Finn GM, et al. Predictive validity of the UKCAT for medical school undergraduate performance: a national prospective cohort study. BMC Medicine. 2016;14(1):140.
  3. Coates H. Establishing the criterion validity of the Graduate Medical School Admissions Test (GAMSAT). Med Educ 2008; 42:999-1006.
  4. Garrud and McManus. UKMED Project - Impact of accelerated graduate-entry medicine courses – Final Report

Doctor Paul Tiffin
pat512@york.ac.uk
UKMEDP054
Declared disability in the UKMED dataset 2002-2016: an exploratory descriptive analysis

Approved on 25 September 2017 UKMED Advisory Board

Published in BMJ Open in April 2022 as "Factors associated with declaration of disability in medical students and junior doctors, and the association of declared disability with academic performance: observational study using data from the UK Medical Education Database, 2002–2018 (UKMED54)"
Disability is an important consideration in the selection of medical students, in undergraduate and postgraduate medical training, and in the practice of medicine. First, adjustments may be required in order to accommodate disabled individuals. Second, disability may affect the fitness of individuals to practise medicine and/or their choice of postgraduate specialty. Third, disability may affect academic performance. Fourth, disability is the subject of legislation (1) which is a key imperative in decisions about disability and how to deal with it. Finally, and related to the previous point, equity with respect to disability and selection is important at both undergraduate and postgraduate level. The General Medical Council (GMC) recognises the importance of disability, providing guidance to medical schools (2). Likewise the Medical Schools Council (MSC) provides advice on how to adjust for disability in the selection of medical students (3). A 2007 report by the British Medical Association (BMA) focused on inequality for disabled doctors and medical students (4). The focus of these documents appropriately reflects the key imperative of relevant legislation, and they provide useful, practical advice on what can be done to accommodate disabled colleagues. Many important questions about disability nevertheless remain unanswered; some of these are outlined in the Research section below. The advent and linkage of the UKMED databases provides the framework for these and other questions to be explored in far more detail than ever before.

References

  1. Equality Act 2010 and Disability Discrimination Act 1995. Although the Equality Act replaced the Disability Discrimination Act, the latter was the key legislative driver during a significant part of the proposed study.
  2. Gateways to the professions: Advising medical schools: encouraging disabled students. GMC (last updated November 2016). Key guidance to medical schools regarding appropriate measures and approaches to declared disability.
  3. Recommendations on selection of medical students with specific learning disabilities including dyslexia. Medical Schools' Council 2005. Generic guidance on the approach to applicants declaring disability.
  4. BMA: Disability equality within healthcare: the role of healthcare professionals (2007). Useful policy document which provided useful sources of further information, e.g. Appendix 5 provides UCAS data on the number of applicants and acceptances to higher education and pre-clinical medicine by disability.

Doctor Michael Murphy
m.j.murphy@dundee.ac.uk
UKMEDP057
Selection Tests as a predictor of acceptance rate and post-graduate success of widening participation students in undergraduate medicine

Approved on 05 November 2017 UKMED Advisory Board

Students from non-traditional backgrounds are underrepresented in UK medical schools. Those that successfully receive a place are also at the greatest risk of non-completion. The impact of specific admissions processes on widening participation (WP) and their ability to select students who will be successful are unclear. We will compare the outcomes of WP medical students across different Medical Schools. Specifically, the study will focus on the use of the Biomedical Admissions Test (BMAT) and UK Clinical Aptitude Test (UKCAT) to determine: 1) The relative proportions of WP students at medical schools using the BMAT or UKCAT as part of their selection criteria. 2) Whether WP students attending BMAT-selecting Medical School perform significantly differently (including degree outcomes and drop-out rates) compared to their non-WP peers and compared to WP/Non-WP students in UKCAT-selecting Medical Schools. 3) The effect of WP status and admissions tests on the proportion of local WP students that they attract. 4) The effect of WP status and admissions tests on the employment of medical graduates; particularly the geographic location of trainee positions and whether they receive their first choice placement. This will provide evidence regarding the utility of BMAT and UKCAT as determinants of success for students from WP backgrounds, and to identify possible relationships between WP status and admissions tests, propensity to attend a local medical school and propensity to return to or remain in a home region following graduation. Understanding these relationships will allow us to make recommendations regarding admissions processes to optimise WP student success.

References

  1. Cleland JA, Dowell J, Nicholson S, Patterson F. (2014). How can greater consistency in selection between medical schools be encouraged? (accessed 21/07/2017). This paper evaluates the various elements of selection methods used by UK Medical Schools and the effectiveness of their predictive validity.
  2. Crawford, C. (2014). Socio-economic differences in university outcomes in the UK: drop-out, degree completion and degree class. Institute for Fiscal Studies Working Paper W14/31 Published 04 Nov 2014 (accessed 21/07/2017). Using accessible data on students pre and post entry to HE, this paper identifies variances in outcomes of WP and non-WP cohorts on the same course, highlighting differences in data returns when controlling for school characteristics rather than socio-economic background.
  3. Holton, M. (2017). Traditional or non-traditional students?: Incorporating UK students' living arrangements into decisions about going to university. Journal of Further and Higher Education. This paper looks at how students’ residential situation whilst studying might affect the support they require for on-course success through a qualitative study drawing on the experiences of a sample of students from a range of accommodation at one institution.

Professor Kevin Murphy
k.g.murphy@imperial.ac.uk
UKMEDP058
Modelling the consultant physician workforce

Approved on 24 February 2018 UKMED Advisory Board

Medical workforce issues are among the most crucial problems facing the NHS (Ref 1 and 2). Some factors contributing to this are: - Fewer international doctors (possibly related to immigration controls and Brexit) - Increased less than full time working within the medical workforce (possibly related to feminisation of the workforce) and - New junior doctors' contract dispute leading to recent industrial action and the exposure of system wide non contractual issues within the trainee workforce linked to low morale and inflexibility of training programmes. The government has pledged to increase medical school places by 1500 beginning September 2018 to attempt to address this complex issue. We plan to construct a Markov mathematical model to model future progression from medical school entry to becoming a consultant physician using UKMED data to provide the numbers moving between different training stages up to higher specialty training (HST). JRCPTB data will be used for progression from HST level. Sensitivity analysis will facilitate targeting of future interventions to increase the number of consultant physicians available in general and in frontline specialities such as acute medicine and geriatric medicine.

References

  1. Underfunded, underdoctored, overstretched: RCP London. 2. Census of consultant physicians and higher specialty trainees 2016-17.

Doctor Johnny Boylan
ohn.boylan@rcplondon.ac.uk
UKMEDP072
Factors associated with working as a locum

Approved on 25 September 2017 UKMED Advisory Board

Little is known about the numbers of doctors who leave training and become locums. This study seeks to establish the proportion of Drs who have worked in the UK as an F2 Dr between 2012 and 2016 and subsequently worked as a locum. We will consider those who worked as locum where there is evidence of failure to progress in training as captured by ARCP and recruitment data and those who worked as locum where there is no evidence of failure to progress in training. We will examine whether working as locum with no previous evidence of failure to progress in training is associated with demographic factors, geographic factors, specialty factors, previous academic performance, working or wanting to work less than full-time (LTFT) and the workload indicator from the final NTS submission prior to working as a locum.

References

  1. A Pubmed search on “locum” did not return any recent research on who works as a locum or why. Research that was obtained is old, with small sample sizes and relies on self-report. The Campbell at al study is of note as it suggests that colleagues give locums lower performance ratings. BMJ. 2011 Oct 27;343:d6212. doi: 10.1136/bmj.d6212. Factors associated with variability in the assessment of UK doctors' professionalism: analysis of survey results. Campbell JL1, Roberts M, Wright C, Hill J, Greco M, Taylor M, Richards S.
  2. Campbell et al found that locum doctors received lower rating from their colleagues on a measure of professional performance obtained using a GMC questionnaire. Can Fam Physician. 2010 May;56(5):e183-90.
  3. Locum practice by recent family medicine graduates. Myhre DL1, Konkin J, Woloschuk W, Szafran O, Hansen C, Crutcher R. In a survey of 152 graduates who had completed family medicine training between 2001 and 2005 and undertaken locum work the authors found female and younger family physicians were more likely to practise as locums. The most common reason for doing so was as a practice exploration to increase experience or competence.
  4. Br J Gen Pract. 1999 Jul; 49(444):519-21. Locum doctors in general practice: motivation and experiences. Questionnaires were returned by 111 doctors currently working as locums in general practice. Four main reasons for working as a locum GP were given: as a short-term option while between posts, to gain experience of different practices before commitment to one practice, to balance work and family or other commitments, and to continue part-time work after retirement.

Mr Daniel Smith
daniel.smith@gmc-uk.org
UKMEDP073
Recruitment of trainees to obstetrics and gynaecology training programmes.

Approved on 04 March 2018 UKMED Advisory Board

Closed

Recruitment to Obstetrics and Gynaecology has shown a large fall in expression of interest in the specialty over time. This has become a significant issue, as more trainees are needed to cover a high-risk speciality such as O&G. Recruitment to O&G varies between medical schools, as does the amount of time spent doing O&G within the curriculum. Factors thought to influence career choice also includes experience as a student. In addition, by the third year after qualification, only 46% of those whose only choice was previously obstetrics and gynaecology were still committed to this career. This research proposal will examine which factors predict recruitment to obstetrics and gynaecology training at undergraduate level and in the early years after qualification. We will consider the following factors: 1. medical school and course type and course level/school level data measuring timing and amount of teaching/exposure students had to obstetrics and gynaecology 2. socio-demographic variables 3. academic attainment on entry and exit from medical school This research proposal will help understand which factors affect recruitment to the specialty and help approach likely challenges ahead. In particular, the aim is to enable exchange of information between medical schools and support areas of development of undergraduate O&G medical education in medical schools where recruitment is low and in addition to aid revision of the current RCOG undergraduate curriculum. Also the information will be used to allow resources to be targeted at areas in the early postgraduate years, which may increase recruitment.

References

  1. Morgan H. How can recruitment be improved in obstetrics and gynaecology? The Obstetrician & Gynaecologist 2011; 13:175–182. This article covers the fall in O&G application over time and explores the reasons for motivation for choosing O&G as a career.
  2. Turner G, Lambert TW, Goldacre MJ, Barlow D. Career choices for obstetrics and gynaecology: national surveys of graduates of 1974–2002 from UK medical schools. BJOG 2006; 113:350–6. This paper looks at trends in career choice for O&G among UK medical graduate and suggests that workforce planning and career progression planning may be attributable, rather than lack of enthusiasm for the specialty.
  3. Alberti H, Randles H, and Robert K McKinley R. Exposure of undergraduates to authentic GP teaching and subsequent entry to GP training: a quantitative study of UK medical schools. Br J Gen Pract 28 February 2017; bjgp17X689881. These researchers explored the correlation between time spent in general practice as an undergraduate and choice of GP training as a career. Time spent as an undergraduate in O&G varies between medical school and it would be interesting to investigate whether this has any correlation with recruitment to the specialty.
  4. Harding A, Rosenthal J, Al-Seaidy M, Pereira Gray D, and McKinley R. Provision of medical student teaching in UK general practices: a cross-sectional questionnaire study. Br J Gen Pract 2015; 65 (635): e409-e417. This study also quantifies current exposure of medical students to general practice and compares it with past provision and also with postgraduate provision.
  5. Cleland, JA, Johnston, PW, Anthony, M., Khan, N. & Scott, NW. A survey of factors influencing career preference in new-entrant and exiting medical students from four UK medical schools. Medical Education 2014; 14: 151. This study shows that medical schools appear to differ in their influence over career preference and recommend that comparisons across medical school populations must control for medical school processes as well as differences in the students when looking at career preference.

Doctor Philippa Marsden
Philippa.marsden@newcastle.ac.uk
UKMEDP077
The relationship between medical student Conscientiousness Index scores and later clinical performance: a pilot study

Approved on 22 May 2018 UKMED Advisory Board

Published in BMJ Open in November 2020 as "‘10% of your medical students will cause 90% of your problems’: a prospective correlational study"
Performance by medical and other healthcare students while in education may provide a guide to their later clinical practice. Demonstrating that a metric has Predictive Validity would therefore allow targeted training to be directed to individuals who may cause concern during their education or permit appropriate selection. It is known that cognitive ability has significant Predictive Validity for later clinical practice in a number of settings. However, there are currently no demonstrated measures of personality related performance with such Validity. Our project involves data captured as part of the Conscientiousness Index (CI) project(1). Data on students’ performance of routine tasks such as attendance and submission of assignments, was assembled to form a single score, known as the Conscientiousness Index. Positive, statistically significant correlations were observed with outcome performance such as examinations scores and independent staff ratings of professionalism. The process has since been repeated in other health care settings(2,3). The CI Scores will represent the Predictor Variable and we wish to explore the relationship with this data with several Outcome Variables as contained in the UKMED database. In order to allow us to draw any conclusion on the predictive validity of the CI for future performance as a doctor, two principal methods of analysis will be used in this pilot: linear regression and data dichotomisation into high and low scoring individuals. The long term benefit is to healthcare in the UK in particular, and potentially worldwide, in establishing the Predictive Validity of a personality trait measure, rather than to individuals.

References

  1. McLachlan J, Finn G, McNaughton RJ The Conscientiousness Index: an objective scalar measure of conscientiousness correlates to staff expert judgements on students’ professionalism. Academic Medicine 2009; 84: 559-65.
  2. M. A. Sawdon, K. Whitehouse, G. M. Finn, J. C. McLachlan, D. Murray. Relating professionalism and conscientiousness to develop an objective, scalar, proxy measure of professionalism in anaesthetic trainees. BMC Medical Education. 17:49.
  3. Kelly M, O'Flynn S, McLachlan J, Sawdon M. The Clinical Conscientiousness Index: a valid tool to explore professionalism in the clinical undergraduate setting. Academic Medicine 2012.

Doctor Marina Sawdon
marina.sawdon@sunderland.ac.uk
UKMEDP081
Junior Doctors’ Training Satisfaction and Progress: Longitudinal Examination of Medical Students Individual Differences, Academic Attainment and Work/Learning Environment

Approved on 22 May 2018 UKMED Advisory Board

Closed

Junior doctors in the NHS are under considerable work pressure (1) which may negatively impact their well-being and professional development. To retain a satisfied and competent work force, it is important to better understand which factors predict junior doctors’ work satisfaction, educational satisfaction, and successful progression through training. The link between work conditions and job satisfaction are widely researched (2) but less is known about how work conditions and individual differences (e.g. personality and learning habits) affect junior doctors’ in the UK satisfaction and training progression. Previous studies have shown doctors’ personality and early career experiences (e.g. stress) predict their perceptions of their workplace four to five years later (3). Surface learning habits and stress can lead to poorer learning and lower academic achievement in medical school (4) which in turn may negatively affect junior doctors’ postgraduate examination performance (5). Poor academic performance in medical school can also result in difficulties securing a training position within a competitive specialty and/or geographic location, with weaker trainees ending up working and learning in more challenging environments. Work conditions and individual differences both predict learning and satisfaction, but we do not know how they affect junior doctors’ work and educational satisfaction and progression through training. Neither do we know whether satisfaction is related to educational outcomes. This research project will investigate how individual differences among medical students and work/learning conditions are linked to junior doctors’ satisfaction and training progression.

References

  1. General Medical Council. Training environments 2017: Key findings from the national training surveys [Internet]. 2017.
  2. Antoniou AS, Davidson MJ, Cooper CL. Occupational stress, job satisfaction and health state in male and female junior hospital doctors in Greece. Journal of managerial psychology. 2003 Sep 1;18(6):592-621.
  3. McManus IC, Keeling A, Paice E. Stress, burnout and doctors’ attitudes to work are determined by personality and learning style: A twelve year longitudinal study of UK medical graduates. BMC Med. 2004;2:1–12.
  4. May, W., Chung, E. K., Elliott, D., & Fisher, D. (2012). The relationship between medical students’ learning approaches and performance on a summative high-stakes clinical performance examination. Medical Teacher, 34(4), e236-e241.
  5. McManus, I. C., Woolf, K., Dacre, J., Paice, E., & Dewberry, C. (2013). The Academic Backbone: longitudinal continuities in educational achievement from secondary school and medical school to MRCP (UK) and the specialist register in UK medical students and doctors. BMC medicine, 11(1), 242.

Miss Asta Medisauskaite
a.medisauskaite@ucl.ac.uk
UKMEDP082
Do measures of doctors’ academic ability and their Situational Judgement Test (SJT) scores moderate the relationship between sources of workplace stress and experiencing workplace burnout?

Approved on 22 May 2018 UKMED Advisory Board

For the first time, the 2018 National Training Survey (NTS) includes a measure of workplace burnout – the workplace-based burnout items from the Copenhagen Burnout Inventory. Based on previous research we hypothesise that higher levels of burnout will be associated with a higher workload, a less supportive environment and poorer clinical supervision. These potential sources of workplace stress are also measured by the NTS. We will explore whether the relationships between the sources of workplace stress and burnout are moderated by the doctors’ academic achievement as measured by their Educational Performance Measure - the EPM decile and their SJT scores. Are more highly performing doctors and those with high SJT scores less likely to experience feeling of burnout when exposed to the same levels of workplace stress?

References

  1. Chambers, C.N L. Frampton, C.M. Barclay, M and McKee, M. (2016). Burnout prevalence in New Zealand's public hospital senior medical workforce: a cross-sectional mixed methods study. BMJ Open. 2016; 6(11): e013947. doi: 10.1136/bmjopen-2016-013947 The authors used the Copenhagen Burnout Inventory to measure burnout. They found higher burnout associated with working in emergency medicine and psychiatry compared to other specialties, working more than 14 consecutive hours and being a woman. The authors claim that their study is the first to report levels of burnout through the Copenhagen Burnout Inventory in a multi-specialty nationwide survey of senior doctors and dentists in any country.
  2. Kristensen, Tage S., Borritz, Marianne, Villadsen, Ebbe, and Christensen, Karl B. (2005). The Copenhagen burnout inventory: A new tool for the assessment of burnout. Work and Stress, 19(3), 192–207. The outcome variable we propose to use here is the work-related burnout sub-scale of the Copenhagen Burnout Inventory (CBI). Kristensen et al note that “according to Schaufeli and Enzmann (1998, p. 71) the Maslach Burnout Inventory (MBI) has been applied in more than 90% of all empirical burnout studies in the world, which almost gives the MBI monopoly status in the field (Maslach & Jackson, 1981, 1986).“ However we could not use the MBI on the NTS because the it is a proprietary measure and all NTS items are public domain.
  3. McManus, I. C., Keeling, A., & Paice, E. (2004). Stress, burnout and doctors' attitudes to work are determined by personality and learning style: A twelve year longitudinal study of UK medical graduates. BMC Medicine, 2:29. McManus and colleagues found that burnout can be predicted by measures of learning style and personality taken 5 to 12 years before the measure of burnout when the doctors were applicants to medical school or were medical students. In this proposed study the SJT measure could come from up to 5 years before the measure of burnout.
  4. Medical Schools Council on behalf of the cross-stakeholder Project Group (May 2012). Improving Selection to the Foundation Programme Final Report of the Parallel Recruitment Exercise. Accessed 19 November 2017. This Improving Selection to the Foundation Programme report notes that one of the target domains that the SJT for selection into the foundation programme aims to measure is “Coping with pressure”. On this basis we expect a relationship between the SJT score and the measure of burnout.
  5. Dimou ,FM, Eckelbarger D. (2016). Surgeon Burnout: A Systematic Review. J Am Coll Surg. 2016 Jun; 222(6):1230-1239. doi: 10.1016/j.jamcollsurg.2016.03.022. Dimou and colleagues reviewed 39 articles measuring burnout in Surgeons published since 2000 that met their inclusion criteria. They discuss commonly reported risk factors for burnout but make no mention of educational attainment, presumably because this was not captured in any of studies they reviewed. This may be a gap in the literature, this will be confirmed when further literature searches are completed.

Mr Daniel Smith
daniel.smith@gmc-uk.org
UKMEDP083
Factors associated with non-standard outcome at Annual Review of Competence Progression (ARCP) of higher trainees within surgical specialities in the United Kingdom.

Approved on 22 May 2018 UKMED Advisory Board

Published in Journal of Surgical Education in July 2021 as "Personal Characteristics Associated with Progression in Trauma and Orthopaedic Specialty Training: A Longitudinal Cohort Study" Published in Bulletin of The Royal College of Surgeons of England in August 2021 as "Personal characteristics associated with progression in general surgery training: a longitudinal cohort study" Published in BMJ Open in February 2022 as "Differences in progression by surgical specialty: a national cohort study."
There are ten surgical specialities recognised within the United Kingdom (UK) National Health Service (NHS) (1). Entry into surgical specialties is competitive: in 2016 competition ratios at national recruitment for surgical specialities ranged from 1.31-6.57 to 1 training place (2.) However, despite competitive entry some trainees leave training before reaching the end of the curriculum. Hampton et al. suggested that between 2008-2012 1.7% of surgical trainees relinquished their training number, with drop-out rates as high as 4.2% in some deaneries (3). However, methodology was poor with data acquired by contacting each deanery directly, with variable and mostly poor response rates. There has also been suggestion from other sources that in surgery, as well as other medical specialties, non completion of training and non standard outcome at ARCP is higher in female trainees and in those from a minority ethnic background (4). For reasons of equality, fairness, workforce planning and finance it is important to investigate factors affecting retention in training and award of non standard outcome at ARCP (5). Identification of these factors will allow further targeted research and support to be given, structures and attitudes modified and supported and remedial action to be taken if and where required, to ensure progression through training for all who are able to display the required competencies described by a curriculum.

References

  1. Surgery and the NHS in numbers — Royal College of Surgeons accessed 28/2/2018. First data collected demonstrating demographics of current surgical workforce in the United Kingdom.
  2. 2016 Higher Speciality recruitment competition ratios. Access 28/2/2018. Most recent recruitment figures for surgical specialities in the United Kingdom.
  3. Hampton T, Greenhalgh R, Ryan D, and Das-Purkayastha P. Female surgical trainee attrition. The Bulletin of the Royal College of Surgeons of England 2016 98:3, 134-137 Paper highlighted not only attrition in surgical trainees generally but that females were more likely to leave training programmes than males.
  4. How Do Doctors Progress Through Training? The General Medical Council (accessed 13/3/2018.) Data published from GMC showing that in other medical specialities that has demonstrated demographic factors have impacted on ARCP outcomes.
  5. The Berwick report; Improving the safety of patients in England. National Advisory Group on the Safety of Patients in England. Accessed 28/02/2018. Specific recommendation to Health Education England regarding providing appropriate training for trainees and workforce provision and planning.

Miss Hannah Boyd-Carson
hannahboydcarson@doctors.org.uk
UKMEDP084
The Sociodemographic Characteristics of Oral and Maxillofacial Surgeons

Approved on 22 May 2018 UKMED Advisory Board

Oral and Maxillofacial Surgery (OMFS) is a unique surgical speciality because it requires two degrees, and this, as well as a number of other factors, increases training costs.[1] The amount that those engaged in the OMFS training pathway must pay out of their own pocket to achieve the mandatory requirements to complete the training has previously been estimated to cost £71,431 - £113,105.[1, 2] Although accelerated three year dentistry degrees provide a small financial cushion for qualified medics who intend to become OMFS surgeons, many years of university fees, debt and lost personal income can be a daunting financial barrier.[3] Training for OMFS is roughly three-to-four time more expensive than other surgical specialties, and surgery as a whole is significantly more expensive than medicine.[1, 4] Owing to the profound financial implications of choosing to follow the training pathways towards becoming a OMF surgeon, it may be hypothesized that due to the expense of OMFS training, only students who have a more financially stable support network are able to undertake this career. We aim to perform a retrospective examination of the pre-medical school socioeconomic demographics of those who enrol within OMFS and progress through training using data from the UKMED database. Using this data, we aim to describe the socioeconomic profile of medical graduates embarking on a career in OMFS. This will provide a great insight to determine whether efforts should be made to improve the financial accessibility of OMFS for potential future trainees.

References

  1. O’Callaghan J, Mohan HM, Sharrock A, Gokani V, Fitzgerald JE, Williams AP, et al. Cross-sectional study of the financial cost of training to the surgical trainee in the UK and Ireland. BMJ open. 2017;7(11):e018086.
  2. Isaac R, Ramkumar D, Ban J, Kittur M. Can you afford to become an oral and maxillofacial surgeon? BMJ. 2016;352:i163.
  3. Chadha A, Dastaran M, Herd MK. The first UK dental undergraduate programme for medical graduates–a student perspective. British dental journal. 2009;206(7):353.
  4. Stroman L, Weil S, Butler K, McDonald CR. The cost of a number: can you afford to become a surgeon? The Bulletin of the Royal College of Surgeons of England. 2015;97(3):107-11.

Mr Declan Murphy
murphy.declan.1994@gmail.com
UKMEDP085
Evaluating the outcomes and impact of less than full time training on the medical workforce

Approved on 22 May 2018 UKMED Advisory Board

The NHS is facing a serious shortage of doctors and hospitals are struggling to fill all their junior doctor posts. More junior doctors are taking career breaks because of concerns around work-life balance and a perceived rigidity within the structure of a medical career. Alongside this, the general interest in part-time working among junior doctors is rising, meaning that the demand for this style of training is likely to increase. Aim of the project: To examine how junior doctors working part-time affects their progression through, and completion of, training to become specialists compared to those working full-time. Method: 1. Demographics (ethnicity, age, sex), exam scores (e.g. A levels, Foundation SJTs) and socioeconomic data (including markers of widening participation) held for approximately 60,000 junior doctors will be compared between full-time and part-time doctors using statistical methods. 2. Time taken to complete specialist training and the rate of securing a consultant post will be compared between full-time and part-time junior doctors within a sample of approximately 2500 junior doctors. Questionnaires will be used to investigate working patterns after completion of training (i.e. full-time or part-time) and reasons for training part-time or full-time. 3. Thirty participants comprising junior doctors and trainers of doctors will be interviewed to understand their experiences of undertaking or providing LTFT training. Findings will be used to make informed recommendations to the leading organisations in charge of training doctors on how to improve part-time working and use it to retain doctors within the profession for the benefit of patients.

References

  1. United Kingdom Foundation Programme Office. (2017). The Foundation Programme Career Destination Report 2016. [Accessed 5th June 2017] The NHS is facing a serious shortage of doctors and the number of junior doctors taking breaks in their careers after their first two (Foundation) years of training has been rising, jumping from 28% in 2011 to 50% in 2016.
  2. Federation of the Royal Colleges of Physicians of the UK. Census of consultant physicians and higher specialty trainees in the UK, 2014–15: data and commentary. London: RCP, 2016. [Accessed 10 November 2017] In recent years NHS hospitals have experienced growing problems recruiting junior doctors into specialty training posts, with falling numbers of junior doctors training to be medical specialists and this has caused significant gaps in the provision of patient care, with some NHS services needing to close because of doctor shortages.
  3. Rich A., Viney R., Needleman S., Griffin A., Woolf K., ‘‘You can’t be a person and a doctor’: the work-life balance of doctors in training – a qualitative study’, BMJ Open, Volume 6, issue 12 (2016); [Accessed 10 April 2017] The challenges of maintaining work-life balance affects junior doctors’ morale, their well-being and their decisions about their careers. LTFT working was felt to be a possible solution to the barriers to work-life balance, especially among female trainees with children.
  4. General Medical Council. Promoting flexibility in postgraduate training. GMC 2017. [Accessed 10 November] In 2017 the General Medical Council, the UK medical regulator, made recommendations to promote existing mechanisms for flexible training and to encourage others to continue to make working arrangements for trainees more flexible, however we do not know how an increase in LTFT training will impact on the workforce and patient care.
  5. Randive, S., Johnston, C., Fowler, A. and Evans, C. (2015). Influence of less than full-time or full-time on totality of training and subsequent consultant appointment in anaesthesia. Anaesthesia, 70(6), pp.686-690. This study by Randive et al is a good example of one of the very few studies which has used objective data from doctors’ training records to study the influence of LTFT working on progression through training and career outcomes after training – both of which are key factors which need to be considered during workforce planning.

Doctor Magdalen Baker
magdalen.baker.13@ucl.ac.uk
UKMEDP087
Do ethnic differences in performance and selection across medical education persist when controlling for prior educational attainment?

Approved on 07 December 2018 UKMED Advisory Board

This study aims to investigate the causes of ethnic differences in performance and selection across every stage of medical education, by examining the attainment and selection outcomes of Black and Minority Ethnic (BME) doctors in comparison to their white peers, taking into account ethnic differences in prior attainment. The lower achievement of BME medical students is widely acknowledged [1, 2]. While BME students enter medical school with slightly poorer A levels [3], the ethnic attainment gap widens at medical school [1], and persists into postgraduate medicine, with BME doctors performing relatively poorly in postgraduate examinations, in recruitment, and in ARCPs [1, 4], and being less likely to get their first choice of Foundation programme [5]. However, no comprehensive study to our knowledge has examined the ethnic attainment gap at each stage of medical education, whilst controlling for prior attainment. McManus and colleagues found that it is more difficult for BME students to enter medical school even when they achieve the same A-level grades as their white peers [3]. This suggests the need to investigate the comparative selection outcomes for ethnic minorities within medical education. A large scale study investigating outcomes at each selection point of the medical education continuum, using more recent UKMED data and more granular ethnic groupings, would be extremely beneficial to our understanding in ethnic differences in selection outcomes. This research would create the most comprehensive picture available of ethnic differences throughout the medical education and training pathway, supporting long-term aims of developing and targeting effective solutions.

References

  1. Woolf, K., Potts, H.W. W and McManus, I.C., 2011. Ethnicity and academic performance in UK trained doctors and medical students: systematic review and meta-analysis. BMJ, 342: d901.
  2. McManus, I. C., Woolf, K., Dacre, J., et al. 2013. The Academic Backbone: longitudinal continuities in educational achievement from secondary school and medical school to MRCP(UK) and the specialist register in UK medical students and doctors. BMC Medicine; 11: 242.
  3. McManus, I.C., Woolf, K. and Dacre, J., 2008. The educational background and qualifications of UK medical students from ethnic minorities. BMC medical education, 8(1): 21.
  4. General Medical Council (GMC). 2015. The state of medical education and practice in the UK. Accessed 1st June 2018.
  5. Kumwenda, B., Cleland, J.A., Prescott, G.J., Walker, K. and Johnston, P.W., 2018. Relationship between sociodemographic factors and selection into UK postgraduate medical training programmes: a national cohort study. BMJ open, 8(6): e021329.

Miss Halima Shah
halima.shah@ucl.ac.uk
UKMEDP088
Investigating associations of post-graduate examination performance with socio-demographic characteristics, performance at medical school, medical school, SJT and foundation school: a focus on first stage examinations of MRCP, MRCGP and MRCPsych

Approved on 07 December 2018 UKMED Advisory Board

This study aims to examine the relationship between performance in the first stage of MRCP, MRCGP and MRCPsych with sociodemographic characteristics, the EPM (decile and additional educational achievements), Situational Judgement Test (SJT), medical school and foundation school. We particularly focus on specialties with a key community component, as well as the MRCP, because candidates, who pursue many other specialties with a community component, begin with the MRCP. Additionally, it is timely to reconsider the study of McManus et al. (2008) who demonstrated considerable variation in MRCP examination performance relative to candidates’ medical school. They concluded that “unexplained differences” at entry to medical school and specific medical school components might explain this variation. However, they were unable to look at whether candidate socio-economic background was a contributory factor. This is critical to investigate given: variation between medical schools in student socio-demographics; increasing policy focus on widening access to medicine; and emerging evidence of a relationship between socio-economic background and specialty choice (which will be reflected in who sits particular postgraduate examinations). Improvements in routine data management and the availability of standard performance measures mean it is now possible to do a more forensic examination of these associations. It is important to look at the educational performance measure (EPM) decile and additional educational achievements separately since the relationship between additional educational achievements and success in postgraduate professional exams remains unknown. This work will improve understanding, inform assessment and selection policy and help inform UK government policy regarding the future of healthcare delivery.

References

  1. Jones M, Hutt P, Eastwood S, Singh S. Impact of an intercalated BSc on medical student performance and careers: a BEME systematic review: BEME Guide No 28. Medical Teacher 2013, 35: e1493-e1510. This review investigated studies which considered the association of undertaking an intercalated BSc and subsequent factors. Whilst they concluded that undertaking a BSc was associated with improved UG performance, it was beyond the scope of the review to consider associations with performance in post-graduate exams. Our current study will investigate this relationship through examination of relationship between exam performance and the additional educational achievements component of the EPM.
  2. Kumwenda B, Cleland JA, Walker K, Lee AJ, Greatrix R. The relationship between school type and academic performance at medical school: a national, multi-cohort study. BMJ Open 2017, 7:e016291. This study examined the relationship between socio-demographic characteristics and EPM ranking, finding that students from state schools were more likely to finish in the highest rank of EPM than those from independent schools. Indicators of socio-economic status (eg receipt of income support or free school meals, parental education, POLAR area of residence) did not predict performance at medical school. The present study provides an opportunity to assess whether these findings are borne out in professional exams.
  3. McManus IC, Elder AT, de Champlain A, Dacre JE, Mollon J, Chis L. Graduates of different UK medical schools show substantial differences in performance on MRCP (UK) Part 1, Part 2 and PACES examinations. BMC Medicine 2008, 6:5. This paper found differential attainment in MRCP examination performance relative to medical school. Our proposed study extends on these analyses to include further pertinent variables and a larger cohort of medical schools (33 as opposed to 19).
  4. Smith DT, Tiffin PA. Evaluating the validity of the selection measures used for the UK’s foundation medical training programme: a national cohort study. BMJ Open 2018, 8: e021918. This study found that whilst EPM decile and SJT scores were associated with completion of the Foundation Programme, additional educational achievements had no bearing. The currently proposed study will examine whether these findings hold in professional exams (MRCP, MRCGP and MRCPsych).
  5. Tyrer SP, Leung W-C, Smalls J, Katona C. The relationship between medical school of training, age, gender and success in the MRCPsych examinations. Psychiatric Bulletin 2002, 26:257-263. This study found success in RCPsych Part 1 and Part 2 examinations was associated with medical school of training (higher pass rates in graduates of UK/Ireland) and age of candidate. It was beyond the scope of this study to consider socio-demographic factors, ethnicity and variations related to medical schools within the UK. Our study will address these questions.

Isobel Cameron
i.m.cameron@abdn.ac.uk
UKMEDP089
The UK Medical Applicant Cohort Study: Applications and Outcomes Study

Approved on 07 December 2018 UKMED Advisory Board

Published in BMJ Open in September 2022 as "Institutional choice among medical applicants: a profile paper for The United Kingdom Medical Applicant Cohort Study (UKMACS) prospective longitudinal cohort study."
In March 2018, 1500 new English medical school places were created with an aim to “widen the social profile of medical students”.(1) Increasing the proportion of doctors from ‘non-traditional’ backgrounds (i.e. from under-represented social groups) is a priority in medical education; however the evidence for how to achieve it is still relatively poor. The United Kingdom’s 38 medical schools differ in how they select and educate students, resulting in considerable variability in outcomes for graduates of different medical schools.(2, 3) In particular, some schools attract and/or accept considerably more non-traditional applicants.(4) It is uncertain why. Most medical selection research examines the technical aspects of selection tests; however understanding how applicant choices affect selection outcomes is also needed since, as one admissions dean put it, “we can only select from those who apply”.(5) Long term follow-up of applicants is necessary to understand how applicant factors predict outcomes. The proposed research is part of a National Institute for Health Research funded study, which aims to understand and improve medical applicant choices and outcomes. In the current study we propose to analyse applications to study medicine in the UK from 2007 to 2017 to establish: • Which medical school and applicant factors predict the combination of medical schools that applicants chose to apply to (the maximum being four); • whether the choices of traditional and non-traditional applicants differ; • whether choices predict the likelihood of receiving at least one offer; • whether choices mediate the relationship between applicant social background and likelihood of receiving at least one offer.

References

  1. Health Education England. New medical schools to open to train doctors of the future. 20th March 2018. Accessed 31st July 2018.
  2. Gale TCE, Lambe PJ, Roberts MJ. Factors associated with junior doctors’ decisions to apply for general practice training programmes in the UK: secondary analysis of data from the UKMED project." BMC Medicine. 2017. 15(1): 220.
  3. McManus IC, Elder AT, Dacre JE, Mollon J, Chis L. Graduates of different UK medical schools show substantial differences in performance on MRCP(UK) Part 1, Part 2 and PACES examinations. BMC Medicine. 2008. 6:5 https://doi.org/10.1186/1741-7015-6-5
  4. Steven K, Dowell J, Jackson C, Guthrie B. Fair access to medicine? Retrospective analysis of UK medical schools application data 2009-2012 using three measures of socioeconomic status. BMC Medical Education. 2016;16(1):11.
  5. Cleland JA, Nicholson S, Kelly N, Moffat M. Taking context seriously: explaining widening access policy enactments in UK medical schools. Medical Education. 2015;49(1):25-35.

Doctor Katherine Woolf
k.woolf@ucl.ac.uk
UKMEDP091
Access to HE qualifications and widening participation in medicine

Approved on 07 December 2018 UKMED Advisory Board

Access to HE Diplomas developed to provide pre-university qualifications for mature learners without the usual secondary educational qualifications such as A-levels. Around 35,000 mature students now take Access to HE diplomas each year and circa 600 on diplomas allied to medicine (Mizon, personal communication) that are offered by eight colleges nationally. Overall, success in entering HE runs at 65-70% for those gaining the qualification (Farmer, 2017), but an initial trawl of the UKMED database has only identified 211 medical students entering with Access to HE qualifications between 2011-6, suggesting that the conversion rate to medicine degree programmes may be substantially lower. At present the majority of UK medical schools do not recognise Access to HE qualifications as a sufficient requirement, and amongst those that do recognition may be restricted to Diplomas from only one or a few of the colleges offering them. One reason is probably a perception that these students may be at greater risk of not completing a medicine programme, though there is evidence that they have considerable persistence in their educational courses (Hinsliff-Smith et al, 2012). A more recent study (Wilkinson et al, 2015) reports positive experiences of Access to HE students entering the Bradford-Leeds medicine course, but no evidence about their likelihood of progressing and completing the programme. This project proposes to improve the evidence base in two ways: (i) establishing more clearly the numbers and demographic profile of students entering medical school each year with Access Diplomas, and (ii) their relative success at medical school (progression & completion, FPAS educational performance measure, FPAS Situational Judgment Test).

References

  1. Farmer J. Mature Access: the contribution of the Access to Higher Education Diploma. Perspectives: Policy and Practice in Higher Education. 2017 Jul 3;21(2-3):63-72.
  2. Hinsliff-Smith K, Gates P, Leducq M. Persistence, how do they do it? A case study of Access to Higher Education learners on a UK Diploma/BSc Nursing programme. Nurse Education Today. 2012 Jan 1;32(1):27-31.
  3. Wilkinson D, Dew C, Storey T, Barber J, Awad Y, Pattison J. Mature student progression to healthcare programmes in HE. University of Leeds, October 2015

Doctor Paul Garrud
paul.garrud@nottingham.ac.uk
UKMEDP097
Investigating Core Medical Trainees’ experience of the training environment and associations with workplace-based assessments, progression, training and examination outcomes from an Equality & Diversity (E&D) perspective.

Approved on 19 February 2019 UKMED Advisory Board

Under the Public Sector Equality Duty conferred by the 2010 Equality Act, it is the statutory responsibility of organisations with a public function (such as Medical Royal Colleges) to collect, analyse and interpret data on the outcomes of their training and assessment programmes (1) and to consider how the nine protected groups defined by the Equality Act are differentially impacted by these programmes. This research / data analysis project looks at these issues in more depth for UK core medical training (CMT) by investigating the progression of trainees as influenced by the quality of their training environment (measured by the JRCPTB’s CMT quality criteria), performance in workplace-based assessments and the bearing these factors have on their training outcomes (measured by non-completion of CMT and ARCP outcomes) and examination outcomes (measured by time taken to pass each part of the MRCP examination and number of attempts). Previous studies (2-5) have confirmed the association of certain factors, such as demographics and recruitment scores, with training and examination outcomes however indicators of the quality of the training environment, and its interaction with the formative assessments that take place there, have not been included in previous CMT research or examined from an equality and diversity perspective. We recognise the size and complexity of the analyses required, however we believe this study is essential to deepening understanding of differential attainment.

References

  1. Promoting excellence: standards for medical education and training postgraduate curricula (2015). The guidance confirms that Medical Royal Colleges are required to conduct data analysis and monitor outcomes for equality and diversity reasons (see for example, R2.5).
  2. Patterson, F., Lopes, S., Harding, S., Vaux, E., Berkin, L. and Black, D. (2017), The predictive validity of a situational judgement test, a clinical problem solving test and the core medical training selection methods for performance in specialty training. Clinical Medicine 17.1 (2017): 13-17. The paper examines the long-term validity of CMT and GP selection methods in predicting performance in the Membership of Royal College of Physicians (MRCP(UK)) examinations. CMT selection methods (interview scores and situational judgement tests) predict performance and have good evidence of validity
  3. Stegers-Jager, K., Themmen, A., Cohen-Schotanus, J., & Steyerberg, E. (2015). Predicting performance: Relative importance of students’ background and past performance. Medical Education, 49, 933–945. The study determines the relative importance of pre‐admission characteristics and past performance in medical school in predicting student performance in pre‐clinical and clinical training. It confirms the importance of past performance as a predictor of future performance in pre‐clinical training, but also reveals the importance of a student's background as a predictor in clinical training performance.
  4. Ludka-Stempień, K (2015). Predictive validity of the examination for the Membership of the Royal Colleges of Physicians of the United Kingdom. Doctoral thesis, University College London. The study investigated the relationships between MRCP(UK) scores and results of seventeen knowledge exams and two clinical skills assessments (including specialty exams and MRCGP), training performance assessment outcomes (ARCP), and cases of licence limitations and erasures. It concluded that MRCP(UK) is a valid exam and that gender, ethnicity, and UK primary medical qualification (PMQ) and UK training had a significant effect on performance in MRCP(UK); UK PMQ played a predominant role among these factors.
  5. Dewhurst NG, McManus IC, Mollon J, Dacre JE, Vale JA (2007). Performance in the MRCP(UK) examination 2003–4: analysis of pass rates of UK graduates in the clinical examination in relation to self-reported ethnicity and gender. BMC Medicine 5: 8-10.1186/1741-7015-5-8. The study observed that pass rates for MRCP(UK) examinations varied with ethnicity and gender.

Doctor Miriam Armstrong
miriam.armstrong@jrcptb.org.uk
UKMEDP098
Understanding progression in psychiatry training

Approved on 22 February 2019 UKMED Advisory Board

Published in BJPsych Open in June 2021 as "Uncovering trends in training progression for a national cohort of psychiatry trainees: discrete-time survival analysis"
Attrition is a major concern within psychiatric training. Based on one study in 2012, just 65.8% of psychiatry trainees in the UK plan to stay in psychiatry.(1) In 2017 the fill rates for many higher specialty programmes in psychiatry (ST4) was lower than 60%.(2) The aim of this study, therefore, is to analyse the factors which could help to explain these high attrition rates. The London School of Psychiatry recently carried out a survey with core medical trainees (CT3) which identified a number of factors related to job satisfaction which were central to the decision to step out of training. The systematic review published in 2017 on the crisis in psychiatry in the UK also identified a number of work and learning related barriers associated with choosing psychiatry as a career; for example, 25-50% of trainees were leaving because of lack of resources or lack of adequate supervision.(3) Moreover, studies on trainees in other specialties revealed that training progression depended on trainees’ sociodemographic characteristics. For example, females were doing better in exams,(4) but were more likely to dropout.(5) While this evidence suggests that work/learning related factors and sociodemographic characteristics are important to successful progression through training, these assumptions have not been tested with a large longitudinal sample investigating various work/learning and sociodemographic factors together and cannot be generalizable. We will use mixed methods to explore the dropout rates in psychiatry training nationally and will investigate the factors which predict if a trainee progresses through training successfully or drops out.

References

Doctor Asta Medisauskaite
a.medisauskaite@ucl.ac.uk
UKMEDP101
Investigating the potential factors that might influence Prescribing Safety Assessment (PSA) scores amongst UK final year medical students and their predictive validity for performance in early postgraduate training

Approved on 17 May 2019 UKMED Advisory Board

Closed

Prescribing is the main approach used by the NHS to treat illness, alleviate symptoms and prevent future disease – around 1.5 billion prescriptions are written annually costing £15 billion [1]. Sub-optimal prescribing is common in both primary and secondary care – approximately 10% of hospital and 5% of general practice prescriptions contain errors – these represent a major threat to public health and impose significant additional costs to the service [1]. While healthcare system factors contribute it is clear that all new prescribers should possess the necessary knowledge and skills to face this challenging environment. These concerns led to the development of the Prescribing Safety Assessment (PSA) by the British Pharmacological Society (BPS) and Medical Schools Council [2,3] to enable final year medical students to demonstrate that they were competent to assume independent prescribing responsibilities. The PSA has been delivered to the graduates of all UK medical schools since 2014 (around 45,000 candidates). There are grounds for believing that the PSA is meeting its original objectives of raising the profile of prescribing skills amongst students and their medical schools, identifying candidates who are less well prepared in this key patient safety area and stimulating learning. There are also some reasons to believe that overall performance in the assessment environment is improving (e.g. improving scores on anchor items, reducing variation between schools). The priority now is to investigate (i) the factors predict PSA performance, (ii) the relationship between PSA performance and other measures of attainment and (iii) the extent to which performance in the PSA predicts subsequent postgraduate performance (predictive validity). Previous studies have linked undergraduate performance to subsequent postgraduate attainment but these have usually involved very broad performance measures [4]. The unique feature of this study is that it will investigate these relationships for an assessment that focuses on a very narrow skillset (prescribing and supervising the use of medicines) that has been the focus of concern. The proposed studies are large and will require complex analysis. However, we believe this study is essential to provide a clearer understanding of the factors that influence PSA scores and how predictive these are of subsequent performance in early medical training. Indeed, a recent external review of the PSA has called for this research to be undertaken [5]. We have brought together a research team with significant expertise, knowledge, resources and data analysis experience required to complete this work.

References

  1. 2. Maxwell SRJ, Cameron IT, Webb DJ. Prescribing safety: ensuring that new graduates are prepared. Lancet 2015;385:579–581. This paper describes the background and rationale for the development of the PSA.
  2. 3. Maxwell SRJ, Coleman JJ, Bollington L, Taylor C, Webb DJ. Prescribing Safety Assessment 2016: Delivery of a national prescribing assessment to 7343 UK final-year medical students. Br J Clin Pharmacol 2017;83:2249–2258. This paper describes the construction and process of delivering the PSA and provides a full report of the outcomes including the pass rate against a pre-specified standard of competence, psychometric measurements, inter school variability and evidence of improvement in performance.
  3. 4. Patterson, F., Lopes, S., Harding, S., Vaux, E., Berkin, L. and Black, D. (2017), The predictive validity of a situational judgement test, a clinical problem solving test and the core medical training selection methods for performance in specialty training. Clinical Medicine 2017;17.1:13–17. This paper examines the long-term validity of postgraduate training selection methods (including a clinical problem-solving test and situational judgement test) in predicting performance in the Membership of Royal College of Physicians (MRCP(UK)) examinations and demonstrated good evidence of predictive validity.
  4. 5. McLachlan JC. Independent review of the Prescribing Safety Assessment. April 2019. This paper provides a thorough review of the strengths and weaknesses of the PSA process and provides a series of important recommendations for research and development. Importantly, Recommendation 3 is “That a Predictive Validity study is retrospectively conducted through UKMED”.
  5. 1. Elliott R, Camacho E, Campbell F, Jankovic D, Martyn St James M, Kaltenthaler E, Wong R, Sculpher M, Faria R, (2018). Prevalence and Economic Burden of Medication Errors in The NHS in England. Rapid evidence synthesis and economic analysis of the prevalence and burden of medication error in the UK. Policy Research Unit in Economic Evaluation of Health and Care Interventions. Universities of Sheffield and York. This paper provides a thorough review of the prevalence and impact of medication errors (including prescribing errors) highlighting their impact on patients and public health as well as the significant economic burden they impose on UK healthcare.

Professor Simon Maxwell
s.maxwell@ed.ac.uk
UKMEDP103
Predictors of postgraduate exam performance in psychiatrists

Approved on 17 May 2019 UKMED Advisory Board

Published in BMJ Journals in September 2020 as "Differential attainment in the MRCPsych according to ethnicity and place of qualification between 2013 and 2018: a UK cohort study"
Previous work has explored the determinants of postgraduate performance in both medical specialities and general practice. In particular there has been considerable interest in the nature and causes of differential attainment in postgraduate educational performance between UK and International Medical Graduates (IMGs) (1-4). However, to date, a detailed analysis has not been carried out in relation to psychiatry. It is interesting to note that some degree of differential attainment in relation to the GP ‘Clinical Skills Assessment’ was noted between UK graduates who described their ethnicity as ‘White’ and those who identified as ‘Black and Minority Ethnic’ (BME). A previous investigation suggested that subtle communication and sociocultural differences may underlie differences in attainment between these two medical graduate groups (5). The effective practice of psychiatry demands excellent communication skills and cultural competence. It is noteworthy that one of the largest differential attainment gaps at Annual Review of Competence Panel (ARCP) between UK Graduates (UKGs) and IMGs was for psychiatry (3). Thus, this study intends to elicit the predictors of psychiatric postgraduate exam performance in both UKGs and IMGs and describe any trends in differential attainment in relation to both graduate group (e.g. UKGs, IMGs) and ethnic self-identification.

References

  1. Patterson F, Tiffin PA, Lopes S, Zibarras L. Unpacking the dark variance of differential attainment on examinations in overseas graduates. Medical Education. 2018;52(7):736-46.
  2. McManus IC, Wakeford R. PLAB and UK graduates’ performance on MRCP(UK) and MRCGP examinations: data linkage study. BMJ. 2014;348.
  3. Tiffin PA, Illing J, Kasim AS, McLachlan JC. Annual Review of Competence Progression (ARCP) performance of doctors who passed Professional and Linguistic Assessments Board (PLAB) tests compared with UK medical graduates: national data linkage study. BMJ. 2014;348.
  4. Esmail A, Roberts C. Academic performance of ethnic minority candidates and discrimination in the MRCGP examinations between 2010 and 2012: analysis of data. BMJ. 2013;347(347):f5662.
  5. Roberts C, Atkins S, Hawthorne K. Performance features in clinical skills assessment: Linguistic and cultural factors in the Membership of the Royal College of General Practitioners examination. London: King's College London; 2014.

Doctor Paul Tiffin
pat512@york.ac.uk
UKMEDP104
How do students on gateway courses progress through medicine, compared to standard entry peers of similar backgrounds?

Approved on 17 May 2019 UKMED Advisory Board

Students from lower socioeconomic backgrounds are still traditionally underrepresented in medicine in the UK (1). Subsequently, there has been a recent increase in the number of medical schools offering placements on Gateway Year (GWY) programmes. Designed as a form of widening participation (WP), GWY programmes offer places based on contextual admissions, requiring students successfully complete a Year 0, or Gateway year, in order to matriculate onto standard entry (SE) medicine programmes. There has been limited research suggesting that students on GWY year programmes have slightly lower retention rates compared to SE cohorts, and that GWY year students typically perform academically lower than SE peers in preclinical years, but then as well in clinical years, compared to SE peers (2, 3, 4). However, such research has been limited in the number of GWY programmes (two, single-site studies) included. Further, the existing research compares progression of GWY to SE students as a whole, which may not account for mediating factors related to being from a WP background. This proposed project aims to better understand progression and retention rates of GWY students throughout medical school by not just comparing outcomes of GWY students to those on SE, but with particular focus on how GWY students compare to students of similar socioeconomic, “WP,” background who have met admissions criteria for SE. By comparing GWY students specifically to similar demographic subsets of SE students, this research might provide better context for understanding progression of GWY students, and WP students, overall.

References

  1. Medical Schools Council: Selection Alliance. Selection Alliance 2018 Report: An update ont eh Medical School’s Council’s work in selection and widening participation. 2018 November.
  2. Garlick Pamela B, Brown Gavin. Widening participation in medicine BMJ 2008; 336 :1111
  3. D’Silva, R., Curtis, S., Cleland, J., Barker, M., Rowland, J. Progression and retention: Are there differences betwen students entering via a Gateway programme and traditional entrants? In: Proceedings of the Association for the Study of Medical Education Annual Scientific Meeting [conference proceedings on the Internet]; 2017; University of Exeter, Exeter.
  4. Curtis, S., Blundell, C., Platz, C., & Turner, L. (2014). Successfully widening access to medicine. Part 2: Curriculum design and student progression. Journal of the Royal Society of Medicine, 107(10), 393–397.

Miss Angelique Duenas
hyad29@hyms.ac.uk
UKMEDP105
Exploring the convergent validity of the Prescribing Safety Assessment (PSA)

Approved on 17 May 2019 UKMED Advisory Board

Closed

The University Clinical Aptitude Test (UCAT), first introduced in 2006 as the UKCAT, aims to discriminate between applicants with the same academic qualifications more fairly, to facilitate widening participation with recruitment from under represented social groups, and to accurately and robustly identify the attributes of a good clinician in potential students. The Prescribing Safety Assessment (PSA) examination was developed jointly by the Medical Schools Council and the British Pharmacological Society to address concerns about the prescribing abilities of newly-qualified doctors. The PSA assessment board has identified research work that can be used to further develop the PSA, one area being the relationship between the UCAT cognitive elements and prediction of PSA scores, especially the calculation items in the PSA (1). The UCAT includes quantitative reasoning questions which may predict the outcome in the calculation item (CAL) section of the PSA. Previously, it has been demonstrated that UKCAT scores demonstrate a statistically significant but modest degree of incremental predictive validity throughout undergraduate training(2). The proposed research will analyse the predictive value of the cognitive elements of the UCAT in relation to the scores achieved in the PSA, particularly the calculation skills items of the PSA. The BioMedical Admissions Test (BMAT) is another method that some medical schools use to select appropriate students for a degree and career in Medicine. Whilst the UCAT is more widely used and the focus of one of the PSA assessment board's research objectives, the ability to assess the link between PSA scores and previous BMAT scores is important for the validity of both assessments. Also, it is currently unknown whether performance on the PSA projects other aspects of postgraduate educational performance. For example, it may be that the PSA is associated with performance on tests of clinical knowledge, or even clinical skills, as evaluated by postgraduate Royal College membership exams.

References

  1. 1. Maxwell SRJ, Coleman JJ, Bollington L,Taylor C, Webb DJ. Prescribing Safety Assessment 2016: Delivery of a national prescribing assessment to 7343 UK final-year medical students. British Journal of Clinical Pharmacology 2017; (83).
  2. 2. Tiffin PA, Mwandigha LM, Paton LW, Hesselgreaves H, McLachlan JC, Finn GM, et al. Predictive validity of the UKCAT for medical school undergraduate performance: a national prospective cohort study. BMC Med [Internet]. 2016 Dec [cited 2019 Mar 28];14(1).

Doctor David Hepburn
david.hepburn@hyms.ac.uk
UKMEDP106
The fairness of the UKFPO post allocation process from the perspective of the patient

Approved on 17 May 2019 UKMED Advisory Board

Published in Health Science Reports in November 2020 as "Could the UK Foundation Programme training post allocation process result in regional variations in the knowledge and skills of Foundation doctors? A cross‐sectional study"
Students graduating from UK medical schools spend two years in the UK Foundation Programme, based in NHS Trusts. Around 7,000 students are allocated to Foundation Programme training posts every year, with allocations based on students’ performance in key assessments. The allocation process prioritises student location preferences at both Foundation School (largely based on UK regions) and NHS Trust (local) level, so the students scoring highest on the assessments get their first choice location. While this seems fair to students, at its limit it would mean that all the best-performing students are allocated to hospital A in region X and all the worst-performing students to hospital B in region Y. This could be perceived as unfair from the perspective of the patient, because it could mean that the quality of care received depends – in part – on patients’ geographical location. This study will therefore explore if there is a notable difference in students’ performance in three key assessments (the Situational Judgment Test, Educational Performance Measure and Prescribing Safety Assessment) by region of allocation and, for the West Midlands, within the region. We will use data for around 35,000 students graduating from UK medical schools between 2014 and 2018. The main analysis will use a one-way analysis of variance (ANOVA) approach to assess for differences in mean scores by Foundation School and we will produce a “heat map” for each outcome.

References

  1. Junior doctors from different regions report differences in satisfaction with their training (GMC, 2018).
  2. The UKFPO reports suggest that there may be differences in the ability of students allocated to different regions, but only the lowest score of a student allocated to each region is provided; we will consider the mean and variability in scores (UKFPO, 2018).
  3. Smith and Tiffin (2018) report that Situational Judgment Test and Educational Performance Measure scores are predictive of Foundation Programme performance, so differences at entry could translate into differences in quality of care for patients.
  4. Cousans et al. (2018) report that Situational Judgment Test and Educational Performance Measure scores are predictive of Foundation Programme performance, focusing on those the low and high end of the performance distribution.
  5. The importance of considering training quality as a potential confounding variable in the relationship between scores at entry and performance in post is highlighted by Archer et al. (2015); combined with the results of the GMC surveys, we might expect that training quality is one of the factors used by students in making their region-preferencing decisions.

Doctor Celia Brown
celia.brown@warwick.ac.uk
UKMEDP107
Can Situational Judgment Test (SJT), Educational Performance Measure (EPM) and/or Prescribing Safety Assessment (PSA) scores predict the likelihood of being sanctioned by the General Medical Council (GMC)?

Approved on 17 May 2019 UKMED Advisory Board

Published in Academic Medicine in June 2021 as "Is Academic Attainment or Situational Judgement Test Performance in Medical School Associated With the Likelihood of Disciplinary Action? A National Retrospective Cohort Study"
The aim of this project is to determine if performance in assessments taken while at medical school can predict whether a doctor will be sanctioned by the General Medical Council. Sanctions can take a number of forms, including of erasure, suspension, conditions, undertakings and warnings; all of which are serious outcomes for the doctor in question. Other evidence suggests that performance in “academic” assessments may be predictive of such outcomes, so we wish to add to the evidence base by considering three national assessments: the Situational Judgment Test, Educational Performance Measure and Prescribing Safety Assessment, for which scores are now available for almost all doctors graduating from UK medical schools and beginning work in the NHS. We will include data for doctors starting the UK Foundation Programme in 2013-18 (from 2014 for the Prescribing Safety Assessment), and consider any sanctions imposed up to five years after graduation. We will therefore analyse data for around 42,000 doctors and 140,000 person-years of “exposure” (time working in the NHS when a sanction could be imposed). Our primary analysis will consider if there is a difference in the mean performance scores in the three assessments between those with and without a sanction. We will also consider if raising the minimum standard of performance required on any of the assessments could reduce the rate of sanctions, by investigating the “dose-response” relationship between scores and the probability of a sanction.

References

  1. Wakeford et al. (2018) report that better performance on the MRCP and MRCGP examinations (which are similar in format to those to be considered in our study) reduces the risk of a GMC sanction, although reverse causality is a potential problem.
  2. Yates and James’ case-control study (2010) finds that poor performance at medical school is a predictor of subsequent GMC sanctions; we now have data from a much larger sample to extend this work.
  3. Smith and Tiffin (2018) used UKMED data to show that the Educational Performance Measure and Situational Judgment Test can predict performance in the Foundation Programme; we will extend this work by considering a different post-graduate outcome and a further assessment taken at medical school (the Prescribing Safety Assessment).
  4. Cousans et al. (2017) compare the Foundation Programme outcomes of a sub-group of high and low performers on the Situational Judgment Test and report that low scorers on the Situational Judgment Test have poorer supervisor ratings.
  5. Unwin et al. (2014) report that male gender is a risk factor for being sanctioned by the GMC, suggesting that it is critical to control for gender as a likely confounding variable in our analysis.

Doctor Celia Brown
celia.brown@warwick.ac.uk
UKMEDP108
The effect of eLearning on acquisition of prescribing skills by medical students

Approved on 17 May 2019 UKMED Advisory Board

We will determine whether eLearning can enhance the skills required for safe prescribing. For this study, SCRIPT will be the eLearning resource being assessed and the Prescribing Safety Assessment (PSA) will be the test of the skills required for safe prescribing. As such, we will compare medical students’ performance in the PSA between the periods before and after access was granted to SCRIPT at their respective medical schools. Since the timing of the introduction of SCRIPT differed across medical schools, the study has a natural stepped-wedge design. As such, analysis will be performed using a Hussey and Hughes approach. If test scores are normally distributed, or can be transformed to normality (e.g. logarithmically), then the analysis will be performed using a general linear model. If normality cannot be achieved, then the scores will be dichotomised into pass/fail, and analysed using a binary logistic regression model. All analyses will be conducted using SPSS version 22 (IBM SPSS Inc., Chicago, IL, USA), with P<0.05 considered significant.

References

Doctor Sarah Pontefract
s.k.pontefract@bham.ac.uk
UKMEDP110
Demographic predictors of performance on Multiple Mini Interviews for selection to undergraduate medical programmes in the UK.

Approved on 20 November 2019 UKMED Advisory Board

Multiple-Mini Interviews (MMIs) are an admission procedure designed to test personal attributes and overcome the difficulty of context specificity inherent in panel interviews (1). They have been demonstrated to have good feasibility, acceptability, validity, and reliability (2). Consequently, they have become increasingly popular in both undergraduate and postgraduate selection. Indeed, in the UK 57 of the 73 undergraduate medical programmes (including all entry routes) use MMIs within their selection procedures (3). Our previous systematic review of MMIs identified that there was insufficient evidence regarding potential bias against groups that are underrepresented in medicine (2). Subsequent studies from North America have identified that there is potential differential attainment on MMIs for applicants from lower socioeconomic backgrounds or Black and minority ethnic groups. A study of five Californian medical schools demonstrated that applicants from disadvantaged backgrounds performed better on panel interviews but poorer on MMIs (4). Another study demonstrated that applicants from lower socioeconomic backgrounds, and applicants from Hispanic/ Latino backgrounds performed worse on MMIs (5). Despite the considerable adoption of MMIs for entry to UK programmes, there has been no multi-institution research into potential bias against underrepresented groups. Given the imperatives to widen access to medicine in the UK, it is important and timely to explore this. This study will seek to identify if certain applicant characteristics are associated with differential performance on MMI. We aim to perform multivariate analyses to identify predictors of performance on MMI at six UK medical schools using data within the UK Medical Education Database (UKMED).

References

  1. Eva KW, Rosenfeld J, Reiter HI, Norman FR. An Admissions OSCE: the multiple mini-interview. Medical Education. 2004; 38: 314-326
  2. Rees EL, Hawarden AW, Dent G, Hays R, Bates J, Hassell AB. Evidence regarding the utility of multiple mini-interview (MMI) for selection to undergraduate health programs: A BEME systematic review: BEME Guide No. 37. Medical Teacher. 2016; 38: 443-455
  3. Medical Schools Council. Entry Requirements. Accessed 03/10/2019 https://www.medschools.ac.uk/studying-medicine/applications/entry-requirements?type=medicine-with-a-gateway-year&page=2
  4. Henderson MC, Kelly CJ, Griffin E, Hall TR, Jerant A, Peterson EM, Rainwater JA, Sousa FJ, Wofsy D, Franks P. Medical School Applicant Characteristics Associated With Performance in Multiple Mini-Interviews Versus Traditional Interviews: A Multi-Institutional Study. Academic Medicine. 2018; 93: 1029-1034
  5. Juster FR, Baum RC, Zou C, Risucci D, Ly A, Reiter H, Millder DD, Dore KL. Addressing the Diversity-Validity Dilemma Using Situational Judgment Tests. Academic Medicine. 2019; 94: 1197-1203

Doctor Eliot Rees
e.rees@keele.ac.uk
UKMEDP111
Does performance at the Intercollegiate Membership of the Royal College of Surgeons examination predict performance at ST3 National Selection across all surgical specialties?

Approved on 29 June 2020 UKMED Advisory Board

Successful completion of the MRCS examination is required for all surgeons prior to application for a higher specialty training post. Candidates who score higher at Part B of the MRCS and who pass at first attempt, score higher at ST3 National Selection for General and Vascular surgery (1, 2) and perform better “on-the-job” (3). However, it is not known how MRCS performance correlates with success at National Selection in other surgical specialties. This may explain why the MRCS is used only as a condition for eligibility to apply for ST3 Selection rather than as a core part of the ST3 National Selection process. This contrasts with other surgical selection processes. For example, in the USA, the USMLE is the most valued factor in the screening process for General surgery residents (e.g., 4). Moreover, the COVID-19 pandemic has led to the suspension of the usual face-to-face 2020 National Selection. Instead, most specialties are recruiting based on candidates’ portfolio self-assessment score (usually a preliminary screening tool to select candidates for interview). There is no evidence of the validity of this approach and much evidence on the limitations of self-assessment of performance (5). Accumulating further confirmation as to the predictive validity of the MRCS will provide an evidence-base for future decision making. We aim to compare Educational Performance Measure (EPM) scores and MRCS scores to ST3 National Selection first attempt scores across all surgical specialties. We hypothesise that EPM and MRCS performance will predict ST3 national selection score for all surgical specialties.

References

  1. Study which compared MRCS success with performance at General and Vascular surgery ST3 National selection, but did not look at performance at ST3 selection in other surgical specialties .
  2. Demonstrated the correlation between MRCS pass marks and attempts with FRCS success, helping to validate the MRCS as a marker of good surgical knowledge and skill.
  3. Study that revealed the correlation between worse MRCS part B scores and increased number of attempts to pass with an increased likelihood of poor ARCP outcomes as a marker of career progression .
  4. Paper demonstrates that USMLE step 1 scores are valued as the most important factor in the preliminary screening of general surgical residents.
  5. This study and many since in the area of self-assessment show that self-ratings are poorly correlated with other performance measures.

Mr Ricky Ellis
ricky.ellis@doctors.org.uk
UKMEDP112
How well in medical school applicants do UCAS predicted grades relate to attained grades in examinations, and how do predicted and attained grades differ in their predictive validity for undergraduate and postgraduate outcomes?

Approved on 28 May 2020 UKMED Advisory Board

Published in medRxiv in June 2020 as "Calculated grades, predicted grades, forecasted grades and actual A-level grades: Reliability, correlations and predictive validity in medical school applicants, undergraduates, and postgraduates in a time of COVID-19" Published in BMJ Open in December 2021 as "Predictive validity of A-level grades and teacher-predicted grades in UK medical school applicants: a retrospective analysis of administrative data in a time of COVID-19"
The Covid-19 pandemic of March 2020 has resulted, for the first time ever, in Level 3 examinations in schools, such as A-levels and SQA assessments, being cancelled. These assessments normally form the core of medical school selection processes, in particular being a key part of the conditional offers received by many applicants. Ofqual in England has said that A-level results will be replaced by ‘calculated grades’, which will be derived from estimated grades and rankings provided by teachers at the end of May 2020 (and broadly similar arrangements are in place in Scotland, with ‘holistic assessments’ to be provided by teachers). In essence these calculated grades and holistic assessments are likely to correlate highly with predicted grades already provided by teachers on UCAS application forms in October 2019. However surprisingly little is published on the details of the relationship between predicted and actual grades, especially in high-achieving applicants such as those for medicine, and almost nothing on the key question of the relative predictive validity of predicted and actual grades. This project is built upon two existing projects, UKMEDP089 and UKMEDP051. UKMEDP089 is a part of the UKMACS study of medical school applicants, which as one of its workstreams is looking at data from UCAS since 2007 on medical school applications, including attained and predicted grades. UKMEDP051 is a project which primarily is looking at the predictive validity of the aptitude tests BMAT, GAMSAT and U(K)CAT, in the context of Level 2 and Level 3 qualifications (e.g. GCSEs and A-levels), in relation to outcomes from undergraduate training (dropout, UKFPO EPM, UKFPO SJT and PSA) and from postgraduate assessments (such as MRCP(UK), MRCS, MRCGP, etc.). The data from P89 in the build of 21/2/2020 will provide detailed data on the relationships between predicted and attained grades, and the data from P51, in the build of 13/5/2019, will allow the analysis of predictive validity in applicant cohorts from 2010 to 2014, with undergraduate and postgraduate outcomes.

References

Professor Chris McManus
i.mcmanus@ucl.ac.uk
UKMEDP113
Predictive Validity of the Multiple Mini-Intervew in the UK: a National Multi-Cohort Study

Approved on 29 June 2020 UKMED Advisory Board

The Multiple Mini-Interview (MMI) has become a mainstay of the medical school selection process in the UK since its introduction in Dundee and St George’s in 2008. Currently, selection to 30 of the 39 standard-entry medical courses includes some form of MMI (MSC, 2020). Early data from individual schools suggest that MMIs can outperform such staples of selection as traditional interviews (e.g. Eva et al., 2004) and the UKCAT (e.g. Husbands & Dowell, 2013). Although valuable, current studies have two major shortcomings: Candidates who scored too low on the MMI to receive an offer are lost to follow-up; therefore, predictive validity for the lower end of the scale remains unknown. In addition, no UK studies have linked MMI with PG exam performance. Including more schools in the project would capture predictive data for a wider range of scores and enable correlation with early PG assessments (see Eva et al., 2012). No comprehensive investigation has examined the availability of common MMI data across the medical schools or the feasibility of a nationwide study of the MMI’s reliability, predictive validity, and fairness. We propose a twofold study. A scoping investigation would determine which types of studies are possible with extant data, and which data are most needed to enable more comprehensive national studies in future. A collaborative proof-of-concept study would be undertaken by Dundee and St. George’s. At both schools, data are available for the cohort of candidates who sat the MMI in 2012 and would have completed medical school by 2018.

References

  1. Eva, K. W., Reiter, H. I., Rosenfeld, J., & Norman, G. R. (2004). The ability of the multiple mini-interview to predict preclerkship performance in medical school. Academic medicine, 79(10), S40-S42.
  2. Eva, K. W., Reiter, H. I., Rosenfeld, J., Trinh, K., Wood, T. J., & Norman, G. R. (2012). Association between a medical school admission process using the multiple mini-interview and national licensing examination scores. Journal of the American Medical Association, 308(21), 2233-2240.
  3. Husbands, A., & Dowell, J. (2013). Predictive validity of the Dundee multiple mini‐interview. Medical education, 47(7), 717-725.
  4. Medical Schools Council (2020). Entry requirements.

Doctor Bonnie Lynch
b.y.lynch@dundee.ac.uk
UKMEDP114
Taking time out of training after the Foundation Programme: implications for application to core, run-through and specialty training and subsequent performance

Approved on 18 December 2020 UKMED Advisory Board

Recent studies in the UK (UK Foundation Programme Office, 2015; Cleland et al, 2019) show that the proportion of foundation doctors year two (FY2) who proceed immediately into core or specialty training has fallen from more than 70% in 2011 to approximately 30% in 2019. The decision of early career doctors to take “time out of training” (i.e. delay entry to core/specialty training) can adversely affect workforce planning for health service providers and potentially the quality of care for service users. However, little is known about longer-term implications of this behaviour in terms of differences in career decisions, success of specialty selection processes, and subsequent performance between doctors who proceed to training posts immediately after FY2 and those who take time out of training. For example, if those who take time out of training alter their behaviour in applying for further specialty training, it could limit the ability of some specialties to choose the best suitable candidates and adversely affect the performance of doctors in training. Conversely, doctors who delay their decision could gather more information on a specialty through, e.g. locum working, and hence choose a more suitable specialty to their preference and ability. Thus, effect on average performance of doctors in core/specialty training and movement through the training pathway remains an empirical question we aim to examine in this project. Subsample analysis by groups of specialties and numbers of applications will be explored, while controlling for confounding factors such as sociodemographic variables and pre-specialty training attitude and performance.

References

  1. Cleland, J., Prescott, G., Walker, K., Johnston, P. and Kumwenda, B., 2019. Are there differences between those doctors who apply for a training post in Foundation Year 2 and those who take time out of the training pathway? A UK multicohort study. BMJ open, 9(11). [This paper explores the determinants of the decision of foundation doctors to delay their core or specialty training including sociodemographic data and past performance in the medical school. Our project looks at taking time out of training from a different perspective, i.e. whether the delaying doctors would have different performance in their training post.]
  2. McManus, I.C. and Wakeford, R., 2014. PLAB and UK graduates’ performance on MRCP (UK) and MRCGP examinations: data linkage study. Bmj, 348, p.g2621. [This paper shows how MRCP examination can be used as a measure for post-graduate’s outcome performance.]
  3. Rizan, C., Montgomery, J., Ramage, C., Welch, J. and Dewhurst, G., 2019. Why are UK junior doctors taking time out of training and what are their experiences? A qualitative study. Journal of the Royal Society of Medicine, 112(5), pp.192-199. [This study provides insights into why increasingly doctors are taking time out of training after finishing foundation programme training.]
  4. Santana, I. R., & Chalkley, M., 2018. A sequential analysis of the specialty allocation process in the UK. Empirical evidence from the UKMED database. [This paper models the application and recruiting process for specialty training in the UK sequentially.]
  5. Scanlan GM, Cleland J, Walker K, et al, 2018. Does perceived organisational support influence career intentions? The qualitative stories shared by UK early career doctors. BMJ Open, 8: e022833. [This study explores how experiences in early postgraduate training affect the career intentions of trainee/resident doctors. So it can provide some qualitative explanation for this project.]
  6. Spooner S, Gibson J, Rigby D, et al, 2017. Stick or twist? Career decision-making during contractual uncertainty for NHS junior doctors. BMJ Open, 7: e013756. [This paper provides insight on reasons why doctors took time out or switched specialty.]
  7. Tiffin, P.A., Illing, J., Kasim, A.S. and McLachlan, J.C., 2014. Annual Review of Competence Progression (ARCP) performance of doctors who passed Professional and Linguistic Assessments Board (PLAB) tests compared with UK medical graduates: national data linkage study. BMJ, 348, p.g2622. [This paper provides us with a practical way to categorise ARCP outcomes and use them as one of our dependent variable.]
  8. Kumwenda B, Cleland J, Prescott G, et al, 2019. Relationship between sociodemographic factors and specialty destination of UK trainee doctors: a national cohort study. BMJ Open, 9:e026961. [This paper demonstrates one way of specialty re-classification that can be applied in this project.]

Doctor Diane Skatun
d.skatun@abdn.ac.uk
UKMEDP115
Performance on Situation Judgment Tests and Risk Fitness of Practise Issues in UK Medical Students

Approved on 29 June 2020 UKMED Advisory Board

Published in Medical Education in March 2022 as "Situational judgement test performance and subsequent misconduct in medical students."
The use of situational judgement tests to evaluate ‘non-academic attributes’ has rapidly been rolled out in medical selection. There is overall meta-analytic evidence that the scores from such tools are generally valid and add value in selection decisions above measures of knowledge or cognitive ability (Webster, Paton et al. 2020). Nevertheless, this latter evidence synthesis highlighted that the validity coefficients from SJT evaluations based on postgraduate settings were statistically significantly larger than those reported for undergraduate settings (β = 0.21, p<0.001). Previously we have demonstrated a statistically significant association between performance on the SJT used in the allocation process for the foundation years and the risk of reporting a conduct related fitness to practice issue at provisional registration with the General Medical Council (Paton, Tiffin et al. 2018). However, these SJTs were administered after medical student selection into undergraduate studies had already taken place. Moreover, as a result of this initial study, changes in the way that fitness to practice declarations were made have been enacted. These are now intended to be validated by medical schools, rather than purely relying on the self-report of the student at provisional registration. The extent to which the self and medical school reports agree will be evaluated shortly by the GMC. The situational judgement test currently used as part of the University Clinical Aptitude Test (UCAT) was first piloted in 2013 and used within selection in 2014. Validation studies highlighted relatively modest, though statistically significant, correlations between the scores on this SJT and subsequent tutor ratings (Patterson, Cousans et al. 2017). An independent analysis also highlighted that the six different forms of the test may have been working slightly differently in the pilot version (Tiffin and Carter 2015). Thus, it appears timely to conduct a study specifically in relation to SJT performance and the risk of undergraduate conduct problems. That is, if SJTs are to be effective in selection then they should have a positive footprint in reduced rates of professionalism lapses in both medical students and doctors. Moreover, it is established that lapses of professionalism in undergraduate studies are a risk factor for subsequent censure for fitness to practice issues in qualified doctors (Papadakis, Teherani et al. 2004). If we report positive findings then it would be clear that the UCAT SJT is serving an important role in helping to ensure public protection from unprofessional behaviours in medical graduates and doctors, which are generally linked to personal qualities, rather than lack of clinical knowledge or skills (Tiffin, Paton et al. 2017). This study is now feasible due to changes in the way that fitness to practice lapses have now been recorded over the last three years, and the maturity of UKMED; there are now data in UKMED for medical students who both completed the UCAT SJT in 2013, 2014 and 2015 and have completed a provisional registration declaration; by October 2020 there are estimated to be around 7000 entrants with SJT scores and FTP declarations present. Therefore, we are now in a position to meaningfully model the potential impact of introducing the SJT into undergraduate medical selection. In particular it would be useful to understand where the optimum, and practical, threshold for performance on the UCAT SJT might lie. Thus, our findings would have clear implications for the way that the SJT is implemented within undergraduate medical selection, both within the UK and Australasia.

References

  1. Papadakis, M. A., A. Teherani, M. A. Banach, T. R. Knettler, S. L. Rattner, D. T. Stern, J. J. Veloski and C. S. Hodgson (2004). "Disciplinary action by medical boards and prior behavior in medical school." New England Journal of Medicine 353(25): 2673-2682.
  2. Paton, L. W., P. A. Tiffin, D. Smith, J. S. Dowell and L. M. Mwandigha (2018). "Predictors of fitness to practise declarations in UK medical undergraduates." BMC Medical Education 18(1): 68.
  3. Tiffin, P. A. and M. Carter (2015). Understanding the measurement model of the UKCAT Situational Judgment Test: Summary Report, The UKCAT Board.
  4. Tiffin, P. A., L. W. Paton, L. M. Mwandigha, J. C. McLachlan and J. Illing (2017). "Predicting fitness to practise events in international medical graduates who registered as UK doctors via the Professional and Linguistic Assessments Board (PLAB) system: a national cohort study." BMC Medicine 15(1): 66.
  5. Webster, E. S., L. W. Paton, P. Es Crampton and P. A. Tiffin (2020). "Situational judgement test validity for selection: a systematic review and meta-analysis." Med Educ.

Doctor Paul Tiffin
paul.tiffin@york.ac.uk
UKMEDP119
Does A-level Chemistry predict undergraduate and postgraduate outcomes?

Approved on 29 June 2020 UKMED Advisory Board

The Medical Schools Council has worked with the medical schools to make selection into medicine more fair, open and transparent while at the same time maintaining the high standards set out by the UK medical regulator. Getting admissions processes right is a resource intensive task that requires delicate calibration but yields long term benefits when the right students are admitted, not lost to attrition, and finally enter the medical workforce. For a successful outcome in medicine, students need more than just academic rigour. The GMC requires future doctors to be competent in a number of areas such as communication skills, professionalism, critical appraisal etc (1). However, robust academic skills are still a necessity. Currently, medical schools use a range of measures to select applicants and different schools will use different measures or a combination or measures. These include educational qualifications such as GCSEs, A-Levels, and Scottish Highers, as well as additional admissions tests (UCAT, BMAT, GAMSAT). The debate for the need of natural sciences as a requirement for entry into medicine has been going on for some time. Almost all of the UK medical schools require applicants to have an A-level in the sciences, either Chemistry or biology. Chemistry is a requirement for entry into 32/38 medical school courses in 20/21 (2). There are mixed findings regarding the predictive validity of A-level Chemistry and performance in medical schools. Students with high grades at A-level Chemistry were found to perform better than those without Chemistry or those with lower A-level Chemistry grades during medical school (3)(4). However, the correlation between medical school performance and A-level grades drop as students progress through the course (5). This is expected as the early years of medicine courses cover the basic sciences, and the academic advantage is lost when moving towards the clinical years of the course. Other conflicting findings show that there is no correlation between those who had previously studied natural sciences and the outcome and performance at medical school (6)(7). It should be noted that most of the previous literature were limited to studies within single medical schools and assessing outcomes at the undergraduate level. Given that the early stages of medical degrees cover basic sciences, it is argued that Chemistry may not be needed for admissions and could act a barrier to widening access. A-level Chemistry attainment is predominantly seen in the white ethnic group (8). With the expanded data in UKMED, we intend to investigate this further by looking at both undergraduate and postgraduate outcomes.

References

  1. Tomilson RW, Clack GB, Pettingale KW, Anderson J, Ryan KC. The relative role of "A" level Chemistry, physics and biology in the medical course. Med Educ. 1977;11(2):103‐108.
  2. “A level attainment by pupil characteristics.” GOV.UK Department for Education
  3. Hughes, P. (2002). Can we Improve on how we Select Medical Students? Journal of the Royal Society of Medicine, 95(1), 18–22.
  4. “Entry requirements for UK medical schools 2020 entry.” Medical Schools Council
  5. MONTAGUE, W. and ODDS, F.C. (1990), Academic selection criteria and subsequent performance. Medical Education, 24: 151-157.
  6. James, D.. and Chilvers, C.. (2001), Academic and non‐academic predictors of success on the Nottingham undergraduate medical course 1970–1995. Medical Education, 35: 1056-1064.
  7. Collins, J.P., White, G.R. and Kennedy, J.A. (1995), Entry to medical school: an audit of traditional selection requirements. Medical Education, 29: 22-28.
  8. NEAME, R.L.B., POWIS, D.A. and BRISTOW, T. (1992), Should medical students be selected only from recent school‐leavers who have studied science?. Medical Education, 26: 433-440.
  9. Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175-191.
  10. Curtis, S., Smith, D. A comparison of undergraduate outcomes for students from gateway courses and standard entry medicine courses. BMC Med Educ 20, 4 (2020).

Mr Peter Tang
peter.tang@medschools.ac.uk
UKMEDP120
Factors predicting which doctors will pursue academic training programmes

Approved on 29 June 2020 UKMED Advisory Board

Despite only a small proportion of the medical workforce comprising clinical academics, formal academic training posts that provide protected research time are limited and thus highly competitive. Understanding the drivers for junior doctors to pursue academic careers, and investigating attrition between different stages of formal academic training (i.e. Academic Foundation Programme to Academic Clinical Fellowship), can help address the underlying recruitment crisis into academic medicine. Selection criteria for formal academic training pathways often comprise prior evidence of academic achievement such as publications and presentations. During medical school, achieving these requires significant time and financial commitment outside of the defined curriculum. Such requirements may therefore disadvantage students from less privileged backgrounds, for example, they may have to spend time working part-time jobs to afford the expenses associated with medical school. We aim to use the UKMED database to analyse the characteristics of doctors choosing to pursue academic careers, and of those who successfully attain these highly sought after academic training posts. In particular, we aim to determine the role of demographic factors, aptitude tests, EPM and SJT scores in predicting future academic careers. Exploring these factors will enable us to identify any barriers to pursuing academic careers, and investigate our hypothesis that there is a need for widening access to academic medical careers. This may form a basis for appropriate interventions to tackle this issue.

References

  1. The MSC calls for continued efforts by employers, funders and related organisations to strive for parity between genders and other minorities in the clinical academic staff workforce. Our study proposes to uncover at which stage inequalities are most prevalent (e.g. if UKCAT score is a significantly associated factor, interventions can be targeted at a Pre-medical student level).
  2. This systematic review of 52 predominantly American studies exploring why women are under-represented in academic medicine concluded that there are substantial gaps in the scientific literature of this topic and a need to evaluate a wider range of countries; therefore our UK-based study looking at predictors of those pursuing academic careers will meet an identified unmet need.
  3. In this survey, males, asians, oxbridge graduates and those with an intercalated degree were more likely to report intending to apply for academic training and only 30% of those intending to apply subsequently did. Our study looks to objectively confirm any such factors as predictive markers.
  4. Authors found the literature about the Academic Foundation Programme is almost entirely positive but is lacking in both critical discourse and higher level evaluation, something our study addresses.
  5. This report concludes that “the health of the UK population depends upon the contribution of clinical academics to teaching, research, innovation and clinical practice, and it is vital that students are attracted into academic careers across the range of specialties”, justifying our third research question.

Doctor Roshni Bhudia
roshni.bhudia1@nhs.net
UKMEDP121
To what extent can the attainment gap between BAME and white students in UK medical schools be explained by performance in admissions tests and is the size of the gap dependent on medical school attended?

Approved on 29 June 2020 UKMED Advisory Board

Published in BMJ Open in December 2023 as "Is the awarding gap at UK medical schools influenced by ethnicity and medical school attended? A retrospective cohort study."
Data from UK medical schools demonstrates that an attainment gap exists between students from Black Asian and minority ethnic (BAME) groups and their white peers.(1) On average, BAME students enter medical school with lower A-Level grades and continue to under-perform relative to their white counterparts throughout their undergraduate and postgraduate training.(1,2,3) As with A-Levels, there is evidence that performance in clinical aptitude admissions tests is predictive of performance in medical school.(2,4,5) There is a scarcity of literature investigating the effect of ethnicity on performance in admissions tests and how this relates to future attainment. Using the UKMED database, we aim to investigate the extent to which the ethnic attainment gap observed in UK medical schools is present on admission as reflected in scores achieved in the Biomedical Admissions Test (BMAT), University Clinical Aptitude Test (UCAT) and Graduate Medical School Admissions Test (GAMSAT). By comparing admission scores with academic outcome measures on exit from medical school (including Educational Performance Measure, Prescribing Safety Assessment, Situational Judgement Test, and obtaining first choice Foundation School), we hope to establish if any attainment gap on admission changes throughout the course of medical school. We then seek to compare these outcomes across different UK medical schools, with the aim to identify whether there is significant variation in the attainment gap between BAME and white students across UK medical schools and if there is evidence that the attainment gap closes or widens in relation to the specific medical school attended.

References

  1. Woolf, K., Potts, H.W. W and McManus, I.C. (2011) Ethnicity and academic performance in UK trained doctors and medical students: systematic review and meta-analysis. BMJ, 342: d901.
  2. McManus, I.C., Woolf, K. and Dacre, J. (2008) The educational background and qualifications of UK medical students from ethnic minorities. BMC medical education, 8(1): 21.
  3. McManus, I. C., Woolf, K., Dacre, J., et al. 2013. The Academic Backbone: longitudinal continuities in educational achievement from secondary school and medical school to MRCP(UK) and the specialist register in UK medical students and doctors. BMC Medicine; 11: 242.
  4. MacKenzie RK, Cleland JA, Ayansina D, Nicholson S (2016) Does the UKCAT predict performance on exit from medical school? A national cohort study. BMJ Open. 2016 Oct 7;6(10):e011313.
  5. Emery JL, Bell JF (2009) The predictive validity of the BioMedical Admissions Test for pre-clinical examination performance. Med Educ. 2009 Jun;43(6):557-64.

Doctor Celia Brown
celia.brown@warwick.ac.uk
UKMEDP122
Prediction of performance in Foundation Year 1 Prescribing Competence Assessment from national undergraduate Prescribing Safety Assessment results: A 5-year study

Approved on 29 June 2020 UKMED Advisory Board

Background - The Prescribing Safety Assessment (PSA) has been developed as a reliable and validated tool designed to demonstrate medical students’ competence in relation to the safe and effective use of medicines. This is often followed by individual NHS Trusts safe prescribing assessments at the start of the Foundation Year 1 (F1). The interrelationships between assessments of safe prescribing competence in the medical education continuum merits investigation as this will enable the optimisation of the assessment process. Aim - This study aims to determine whether there is a correlation between scores in the PSA and the South Thames Foundation School regional prescribing assessment (RPA) for F1 doctors. In addition, the influence of a formative versus summative PSA on performance and agreement between the two tests in pass/fail decisions will be investigated. Methods - This is a retrospective data linkage study in which individual scores from the PSA will be identified and matched to scores from the RPA for F1 doctors starting in the South Thames Foundation School region in August 2015 to August 2019. Thereafter, the data will be pseudonymised. The actual data linkage will be performed on password protected university computers and researchers involved in the data linkage will not be involved in the analysis of the linked data. Correlation coefficients will be calculated to assess the relationship, if any, between candidates’ scores in the PSA and RPA. In addition, the Kappa coefficients will be used to determine agreement between the two tests in pass/fail decisions.

References

Doctor Michael Okorie
m.okorie@bsms.ac.uk
UKMEDP123
Socio-economic variance in surgical training outcomes

Approved on 19 October 2020 UKMED Advisory Board

Widening access programmes are well established in many professional domains in the UK. However, there has been a limited increase in entrants to medicine from low-income areas (1). In order to provide the best possible care for patients, doctors, including surgeons, should be representative of the population they serve (2). Those from low-income backgrounds are significantly less likely to enter a surgical specialty (3,4). Additionally, surgical specialties are the most expensive in terms of meeting required and desired criteria, and the average additional cost of reaching the end of training is £40,000 (5,6). We carried out a recent study, in which we surveyed 278 potential or current surgical trainees, the results of which suggested the cost of surgical training was disproportionately affecting those from low-income backgrounds. Building on this initial analysis, our aim is to assess socio-economic variables for surgical trainees within the UK, using data acquired from the UKMED database, and compare these to reference groups. Additionally, we will assess performance-related measures, which include post-graduate RCS membership exams and interview scores, to assess for any relationship with socio-economic background.

References

  1. 1. Medical Schools Council, 2018. Selection Alliance 2018 Report. An update on the Medical Schools Council’s work in selection and widening participation.
  2. 2. HM Government, 2012. Fair Access to Professional Careers: A progress report by the Independent Reviewer on Social Mobility and Child Poverty
  3. 3. Kumwenda, B., Cleland, J., Prescott, G., Walker, K., Johnston, P., 2019. Relationship between sociodemographic factors and specialty destination of UK trainee doctors: a national cohort study. BMJ Open 9, e026961.
  4. 4. Rodriguez Santana, I., Chalkley, M., 2017. Getting the right balance? A mixed logit analysis of the relationship between UK training doctors’ characteristics and their specialties using the 2013 National Training Survey. BMJ Open 7, e015219.
  5. 5. Stroman, L., Weil, S., Butler, K., McDonald, C., 2015. The cost of a number: can you afford to become a surgeon? Bull. R. Coll. Surg. Engl. 97, 107–111.
  6. 6. O’Callaghan, J., Mohan, H.M., Sharrock, A., Gokani, V., Fitzgerald, J.E., Williams, A.P., Harries, R.L., 2017. Cross-sectional study of the financial cost of training to the surgical trainee in the UK and Ireland. BMJ Open 7, e018086

Mr Zak Vinnicombe
zak.vinnicombe@nhs.net
UKMEDP128
Evaluation of Situational Judgment Tests and professionalism of medical students during undergraduate medical education.

Approved on 31 May 2024 UKMED Advisory Board

Ongoing Project

Internationally, regulatory bodies mandate professionalism behaviours in medical trainees and this has become a core aspect of medical school curricula. Situational Judgement Tests (SJTs) are validated tools when assessing ‘non-academic’ attributes after qualification as a doctor and form part of the selection process into medical school (UKCAT) and foundation training within the United Kingdom (UKFPO SJT). Previous studies have shown inverse associations between medical undergraduate SJT scores and professionalism lapses.1,2 A crucial aspect of SJTs is to ensure appropriately professional medical students become doctors; evidence has highlighted doctors with professionalism lapses had associations with medical school professionalism lapses, although such research, utilising a case-control design, was methodologically limited.3,4 Another study, relying on Fitness to Practice self-declaration data, also found a similar association but was limited by the lack of data about unprofessional behaviours during medical school to corroborate findings. 2 To our knowledge there is currently little evidence assessing the longitudinal associations between SJTs taken at various points across the medical career pathway. Crucially, there is also no evidence for the prospective associations between UKCAT and UKFPO SJTs for professionalism lapses exhibited by medical trainees during medical school. This new research will build on the evidence-base by prospectively determining the associations between SJTs at different time-points of medical training, and between these SJTs and unprofessional behaviours exhibited during medical school.2,5 This will provide educationally impactful information to inform medical schools internationally regarding the value of SJTs in identifying unprofessional students in medical school. (243 words)

References

  1. 1. Sahota G.S. and Taggar J.S. (2020) The association between Situational Judgement Test (SJT) scores and professionalism concerns in undergraduate medical education, Medical Teacher
  2. 2. Paton, L.W., Tiffin, P.A., Smith, D. et al. Predictors of fitness to practice declarations in UK medical undergraduates. BMC Med Educ 18, 68 (2018).
  3. 3. Norman G. Identifying the bad apples. 2015Adv Health Sci EducTheory Pract. 20(2):299-303. Norman challenges the case made by Powis and others regarding non-academic attributes and models the sensitivity and specificity of tests required to screen out doctors who might later fall foul of FtP concerns.
  4. Papadakis M, Hodgson C, Teherani A,Kohatsu N. 2004 Unprofessional Behavior in Medical School Is Associated with Subsequent Disciplinary Action by a State Medical Board. Acad Med 79: 244-249.
  5. 5. Smith DT, Tiffin P. Evaluating the validity of the selection measures used for the UK’s foundation medical training programme: a national cohort study. BMJ Open 2018;8:e021918.

Doctor Gurvinder Sahota
gurvinder.sahota@nottingham.ac.uk
UKMEDP129
Predictors of UK medical trainees’ burnout during the COVID-19 pandemic compared to predictors of burnout in the previous two years as measured on the National Training Survey

Approved on 18 December 2020 UKMED Advisory Board

Since 2018 The GMC’s National Training Survey (NTS) has included the Work-related burnout scale from the Copenhagen Burnout Inventory (CBI) [1]. Due to the COVID-19 pandemic the 2020 NTS was slightly different: it ran at different time of year and completion was not mandatory. However, there are 16,536 cases with a 2020 CBI score and a score from at least one of the two preceding surveys. Previous research has suggested that the extent to which doctors experience burnout may “reflect stable, long-term individual differences in doctors themselves” [2]. In this study, we will be able to control for respondent’s burnout as reported in previous years when examining which attributes of posts such as specialty and workload are predictive of higher burnout rating. Burnout is known to vary by specialty [3] but the pandemic may have exacerbated those differences due its varying impact upon specialty workload. Reviews of the literature report little agreement in estimates of prevalence or appropriate scale cut scores due to different definitions and assessment measures [4]. This study would seek to publish a substantive appendix with descriptive statistics for UK trainees by post specialty and other relevant variables. This would include demographic variables such as ethnicity. Variables that are significant will be included in our model of predictors of CBI, together with the NTS measures. We will explore whether the relationships between the sources of workplace stress and CBI scores are moderated by the doctors’ UK medical school performance as measured by their Educational Performance Measure - the EPM decile and their SJT scores. Are more highly performing doctors and those with high SJT scores less likely to experience feelings of burnout when exposed to the same levels of workplace stress? This previously approved UKMED project UKMEDP82 will be included in this piece of work.

References

  1. 1. Kristensen T, et al. The Copenhagen Burnout Inventory: A new tool for the assessment of burnout. Work and Stress 2005;19(3):192-207.
  2. 2. McManus IC, Keeling A, Paice E. Stress, burnout and doctors' attitudes to work are determined by personality and learning style: A twelve year longitudinal study of UK medical graduates. BMC Med 2004;2(1):29.
  3. 3. Shanafelt TD, West CP, Sinsky C, et al. Changes in burnout and satisfaction with work-life integration in physicians and the general US working population between 2011 and 2017. Mayo Clin Proc. 2019;94(9):1681-94.
  4. 4. Rotenstein LS, Ramos MA, Torre M, et al. Prevalence of Depression, Depressive Symptoms, and Suicidal Ideation Among Medical Students: A Systematic Review and Meta-Analysis. JAMA. 2016;316(21):2214–2236.

Doctor Will Carroll
will.carroll@nhs.net
UKMEDP134
What factors lead to differential attainment in obtaining higher surgical training (ST3) posts in the United Kingdom?

Approved on 09 February 2021 UKMED Advisory Board

Core Surgical Training (CST) is the first hurdle in surgical training in the UK for most surgical specialities. This 24-month-long programme enables surgical trainees to rotate through various specialities before deciding to apply for further surgical training. CST continues to be highly competitive with many more applicants than available posts each year. In 2019, there were a total of 1896 applicants for only 648 posts, resulting in an overall competition ratio of 2.93 [1]. The next step after CST is typically applying for higher surgical training (ST3) posts, but there is a paucity of data on the demographic, socioeconomic and education factors that play a role in determining who a) applies for ST3 posts and b) is successful in obtaining ST3 posts. Firstly, it is worth noting that there is an obvious disparity between the sexes when it comes to surgical training. Given that 59% of medical students 54% of foundation trainees are women, it is surprising to see that upon entering the next stage of training, only 41% of those that secure CST posts are women [2,3]. Even more surprising is the fact that this number drops to as low as 12% for female consultant surgeons [3]. Therefore, this investigation aims to look at the gap between successfully securing a CST post and successfully becoming a surgical specialty trainee (ST3). Moreover, a previous study that focused on progression from CST into specialty surgical training revealed that posts are more likely to be offered to recently qualified, UK-trained, white, male applicants; however, this was published in 2011 [4]. Our investigation aims to focus on non-run-through specialities (General Surgery, Otolaryngology, Paediatric Surgery, Plastic Surgery, Trauma & Orthopaedics, Urology and Vascular Surgery) and provide an update to the existing literature by investigating the role that demographic, socioeconomic and educational factors play in applying for an ST3 post and in successfully obtaining an ST3 post. Our aim for wanting to investigate these factors is two-fold: to describe current population characteristics of applicants who apply to and successfully obtain higher surgical training (ST3) posts, and to guide ongoing work to increase diversity and inclusion in surgery. We would like to utilise a similar methodology to two studies: one by Woolf et al. [5] on the effect of sex on speciailty training application outcomes, which was powered by the UK Medical Education database; and the approved UKMED application by Dr Aditi Das, titled “What factors lead to success in obtaining an ophthalmology specialty training (OST) post and completing postgraduate ophthalmology examinations?” (UKMEDP039).

References

  1. Health Education England. Specialty Recruitment Competition Ratios 2019. 2020.
  2. Moberly T. Number of women entering medical school rises after decade of decline. BMJ. 2018 360:k254.
  3. Moberly T. A fifth of surgeons in England are female. BMJ. 2018 Oct 30;363:k4530.
  4. Carr AS, Munsch C, Buggle S, Hamilton P. Core surgical training and progression into specialty surgical training: how do we get the balance right? Ann R Coll Surg Engl (Suppl). 2011: 93:244–248.
  5. Woolf K, Jayaweera H, Unwin E, Keshwani K, Valerio C, Potts H. Effect of sex on specialty training application outcomes: a longitudinal administrative data study of UK medical graduates. BMJ Open. 2019;9(3):e025004.

Mr Siddarth Raj
siddarth.raj@kcl.ac.uk
UKMEDP135
Falling or stepping off the ladder? A comparison of attrition rates of UK medical and surgical postgraduate training schemes. A secondary analysis of the UKMED database.

Approved on 09 August 2021 UKMED Advisory Board

The loss, or attrition, of trainee doctors from postgraduate training schemes is a major issue facing the NHS; threatening the future of speciality training and patient safety (O’Sullivan et al. 2020; General Medical Council 2019). Evidence suggests, 30% of trainees were leaving postgraduate Obstetrics and Gynaecology training in 2017, with the problem worsening (Royal College of Obstetrics and Gynaecology 2017). 17% of all doctors below retirement age were considering leaving the UK Medical workforce in 2019 (General Medical Council 2019). The loss of this number of doctors would be catastrophic to the NHS. Compounding this, 36% of all doctors plan to reduce their hours in the next year and doctors are taking longer to complete their training (General Medical Council 2020). Only 13% of Psychiatry and 53% of General Practice trainees complete training in the minimum time (General Medical Council 2020). These are both forms of attrition that need accounting for as part of workforce planning (Royal College Of Paediatrics and Child Health 2016; General Medical Council 2020). Whilst there has been major investment in improving recruitment into postgraduate training, little research has been done on retention (Barras and Harris 2018). This is despite calls for research on retaining doctors (General Medical Council 2019). This study will examine the rate of attrition in different medical and surgical specialities. This will allow, for the first time, a comparison of training schemes that have the highest and lowest attrition rates and start to identify factors which may be associated with attrition rates.

References

  1. O’Sullivan, M., Newell, S., Frost, J. and Mountfield, J. (2020). The key features of highly effective training units in obstetrics and gynaecology. The Obstetrician & Gynaecologist 22:147–154.
  2. General Medical Council (2019). The State of Medical Education and Practice in the UK 2019 [Online - accessed Nov 2020]
  3. Royal College of Obstetrics and Gynaecology (2017). Staying Power - O&G Magazine Winter 2017 [Online - accessed Oct 2020].
  4. Barras, C. and Harris, J. (2018). Psychiatry recruited you, but will it retain you? Survey of trainees' opinions. The Psychiatrist 36:71–77.
  5. General Medical Council (2020). The State of Medical Education and Practice in the UK 2020 [Online - accessed May 2021]
  6. Royal College Of Paediatrics and Child Health (2016). Royal College of Paediatrics and Child Health MMC Cohort Study (Part 4) [Online - Accessed Nov 2020]

Doctor Steffan *Huw* Prosser Evans
x2i44@students.keele.ac.uk
UKMEDP136
The (Un)Level Playing Field: How Objective Educational Tracking Leads to Unequal Access

Approved on 25 June 2021 UKMED Advisory Board

Educational tracking allocates students into different programs, usually based on standardised test performance. Socioeconomically disadvantaged students must overcome initial gaps through higher personal ability to meet entry criteria for the upper tracks (Morgan et al., 2013). Theoretically then, disadvantaged students tend to be more able than privileged students assigned to the same track. Many UK medical schools use prior educational attainment (GCSEs and A levels) and objective admission tests (UKCAT, BMAT) to select students (Garrud & McManus, 2018). The UKMED data would allow us to compare the performance of students with different backgrounds before and after entering medical school, while controlling for their admissions credentials. We plan to show that due to selection, disadvantaged students have fewer opportunities to enter the upper-level medical programs; however, once they enter medical schools, disadvantaged students will outperform their privileged peers in each track. Previous studies have reported results consistent with our theory but have attributed these to untapped potential rather than selection. Kumwenda et al. (2017) show that students from state-funded schools, i.e. disadvantaged students, with similar pre-entry test scores outperform students from independent schools. Mwandigha et al. (2018) find that, controlling for A-level attainment, students from lower performing secondary schools achieve better undergraduate examination results. Our theory shows that pure selection can drive such a difference within each medical school; however, unlike previous explanations, it predicts that this advantage will be smaller across the overall medical student population. Novelly, our selection effect should manifest for proxies of disadvantage beyond secondary school quality.

References

  1. Curtis, S., & Smith, D. (2020). A comparison of undergraduate outcomes for students from gateway courses and standard entry medicine courses. BMC medical education, 20(1), 1-14. This paper argues that gateway courses support disadvantaged student by cutting down their entry requirements, which coincides with our policy implication.
  2. Garrud, P., & McManus, I. C. (2018). Impact of accelerated, graduate-entry medicine courses: a comparison of profile, success, and specialty destination between graduate entrants to accelerated or standard medicine courses in UK. BMC medical education, 18(1), 1-16. This paper provides an overall description of the UK medical schools selection and training process, and types of medical schools.
  3. Kumwenda, B., Cleland, J. A., Walker, K., Lee, A. J., & Greatrix, R. (2017). The relationship between school type and academic performance at medical school: a national, multi-cohort study. BMJ open, 7(8). This paper has the results supporting our mechanism, showing that students from state-funded schools (disadvantaged) tend to outperform students from independent schools (privileged) once entering medical programs.
  4. Morgan, J., Triebs, T., & Tumlinson, J. (2013). The (Un) Level Playing Field: How Color-Blind Educational Tracking Leads to Unequal Access. Available at SSRN 2266242. This paper is our theoretical framework.
  5. Mwandigha, L. M., Tiffin, P. A., Paton, L. W., Kasim, A. S., & Böhnke, J. R. (2018). What is the effect of secondary (high) schooling on subsequent medical school performance? A national, UK-based, cohort study. BMJ open, 8(5). This paper has the results supporting our mechanism, showing that students from poorly performing secondary schools tend to outperform those from better performing secondary schools.

Doctor Justin Tumlinson
j.tumlinson@lboro.ac.uk
UKMEDP137
The importance of prior academic achievement, demographics and socio-economic characteristics for successful training progression amongst early career anaesthetists

Approved on 04 June 2021 UKMED Advisory Board

Although recruitment to anaesthetics at core training level is highly competitive, with close to 100% fill rates, attrition between core and specialty training is rising (1). Eligibility to apply for ST3 posts is dependent upon timely success in the primary FRCA, and successful completion of the ACCS/Core Training programme. However, more than 40% of CT2s fail to achieve this within the expected two years (2). Understanding which factors predict successful progression through core training is therefore key to designing training programs, may guide efforts towards reducing attrition between core and specialty training, and help address issues related to differential attainment. Anaesthetics post-graduate examinations are perceived as more difficult when compared to other specialties (3). Detailed knowledge of basic sciences is emphasised in the primary FRCA, with content additionally covering topics such as physics and clinical measurement. Self-perceptions of aptitude in physics may therefore influence decision to apply for anaesthetics training, as well as performance in the primary FRCA. Previous studies have demonstrated a correlation between A-level Chemistry and Biology and medical school performance (4) and, more broadly, that prior academic attainment is a predictor of future attainment in medicine (5). However, to date, no study has investigated the importance of prior aptitude in the sciences, demographics, and prior education for a career in anaesthetics. Our study seeks to explore which factors contribute to choice of a career in anaesthetics and subsequent successful progression through core training, including success in the primary FRCA. Exploring these factors will help doctors to make realistic career choices, and provide guidance for preparation for applications to anaesthesia, training in anaesthesia, and the primary FRCA examination.

References

  1. Montague W and Odds FC. Academic selection criteria and subsequent performance. Medical Education 1990, 24: 151-157. This paper found a correlation between performance in the medical course and performance in A-level Chemistry and Biology, but not Mathematics or Physics.
  2. McManus, I., Woolf, K., Dacre, J. et al. The Academic Backbone: longitudinal continuities in educational achievement from secondary school and medical school to MRCP(UK) and the specialist register in UK medical students and doctors. BMC Med 11, 242 (2013). This paper found attainment at secondary school to predict performance in undergraduate and post-graduate medical assessments.
  3. Workforce Data Pack 2018. RCoA, 2018. This document highlights the falling national fill rate in ST3 posts; the fill rate in 2017 was 86.18% compared to 93.81% in 2015.
  4. 2021 Anaesthetics Curriculum, RCoA, 2020. The length of ACCS training has been increased from 3 years to 4, and Core training from 2 years to 3. The College cites the proportion of trainees not completing core training within the expected timeframe as a key driver for this.
  5. A report on the welfare, morale and experiences of anaesthetists in training: the need to listen. RCoA, 2017. Highlights trainee perceptions regarding the FRCA, and notes it as a particular stressor during training.

Doctor Krishma Adatia
krishma.adatia@nhs.net
UKMEDP142
Is the current system of fitness to practise fit for purpose? Exploring the relationship between ethnicity, gender, socio-economic status and disability with fitness to practise declarations and outcomes

Approved on 25 June 2021 UKMED Advisory Board

Medical students and doctors are expected at all times to demonstrate fitness to practise (FtP) [1]. However, there are increasing questions over the disparities seen in FtP declarations and outcomes. White medical students self-declare more FtP misconduct than students from racially minoritised backgrounds (BME) [2], yet UK BME doctors experience higher rates of complaints and sanctions against them, the reason behind this being unclear [3]. Access to new FtP data allows an exploration of differences between self-reported and objective records of misconduct and in the reporting of certain types of FtP concerns. Additionally, current evidence is based on ‘white’ versus ‘non-white’ students. A more granular study of ethnicity will allow better understanding. Furthermore, racial concordance is often a key factor in patient satisfaction [4], and it is unknown whether regional variation in ethnicity results in increased FtP complaints against BME doctors. 77% of medical students with disabilities report discrimination or being labelled as less competent [5]. As a result, many do not disclose their disability, raising concerns students with disabilities may not declare misconduct due to fears over possible career implications. Data also suggests socioeconomically disadvantaged students declare health-related FtP more frequently, and the possible relationship between health and conduct-related FtP remains unexplored [3]. Male doctors receive twice the complaints and sanctions compared with female doctors but it is not known if the types of FtP concerns varies between genders [3]. This study aims to explore these important questions, striving to address potential biases present in the FtP regulatory system.

References

  1. GENERAL MEDICAL COUNCIL. Professional Behaviour and Fitness to Practise.
  2. PATON, L. W., TIFFIN, P. A., SMITH, D., DOWELL, J. S. & MWANDIGHA, L. M. 2018. Predictors of fitness to practise declarations in UK medical undergraduates. BMC medical education, 18, 1-14.
  3. GENERAL MEDICAL COUNCIL 2013. The State of Medical Education and Practice in the UK: 2013, General Medical Council.
  4. RIMMER, A. 2020. Disabled doctors are not getting the workplace adjustments they need, research finds. BMJ, 370, m3189.
  5. TAKESHITA, J., WANG, S., LOREN, A. W., MITRA, N., SHULTS, J., SHIN, D. B. & SAWINSKI, D. L. 2020. Association of Racial/Ethnic and Gender Concordance Between Patients and Physicians With Patient Experience Ratings. JAMA Netw Open, 3, e2024583.

Doctor Renee Ewe
r.ewe@imperial.ac.uk
UKMEDP143
Admissions test score variation and stability

Approved on 25 June 2021 UKMED Advisory Board

Medical schools utilise different aptitude tests, sometimes different tests for different courses, and deploy the scores in a variety of ways (e.g. setting a cut-off, weighting a test score together with other measures, using test scores as a decider between tied candidates). Previous UKMED projects have examined the associations between aptitude test scores and the progress, career choices, and attainment of applicants, medical students, and qualified doctors (e.g. Mwandigha et al, 2018; Garrud & McManus, 2018). However, there is less published research about the aptitude test scores associated with successful application to medicine although the qualifications of entrant students are widely published (see https://www.hesa.ac.uk/support/tools-and-downloads/unistats). The test providers publish annual reports (e.g. https://www.ucat.ac.uk/about-us/technical-reports/ ) but these do not contain any information about how scores differ between successful and unsuccessful applicants or between medical schools and course types. This project proposes to examine how test scores vary between successful and unsuccessful applicants, different course types, and medical schools (including the different ways scores are deployed). It also plans to examine the variation across years and how this is linked to applicant numbers, both nationally, and to the specific medical school or course. The output will be a published report that describes the historical patterns of applicant and entrant aptitude test scores accompanied by analysis of the context (year, applicant numbers, course type, and medical school).

References

  1. Garrud, P. and McManus, I.C., 2018. Impact of accelerated, graduate-entry medicine courses: a comparison of profile, success, and specialty destination between graduate entrants to accelerated or standard medicine courses in UK. BMC medical education, 18(1), pp.1-16.
  2. Higher Education Statistics Agency (HESA). 2021. Unistats dataset.
  3. Maxfield, M., Schweitzer, J. and Gouvier, W.D., 1988. Measures of central tendency, variability, and relative standing in nonnormal distributions: alternatives to the mean and standard score. Archives of physical medicine and rehabilitation, 69(6), pp.406-409.
  4. Mwandigha, L.M., Tiffin, P.A., Paton, L.W., Kasim, A.S. and Böhnke, J.R., 2018. What is the effect of secondary (high) schooling on subsequent medical school performance? A national, UK-based, cohort study. BMJ open, 8(5).
  5. UCAT 2021. Technical Reports

Doctor Paul Garrud
paul.garrud@nottingham.ac.uk
UKMEDP147
The association between exceptionally low scoring Situational Judgement Test results and Foundation Programme training outcomes.

Approved on 08 August 2021 UKMED Advisory Board

The Situational Judgement Test (SJT) was introduced in 2012 and contributes to 50% of the overall candidate score as part of the allocation process to the UK Foundation Programme. The Recruitment Delivery Group (RDG) of the UK Foundation Programme Office (UKFPO) reviews the scores of all candidates sitting the SJT and agrees on a cut-off point to identify exceptionally low scoring candidates (around 3.5 standard deviations below the mean score). Candidates with exceptionally low scores are withdrawn from the recruitment process and invited for an interview-based review process. This is on the basis that exceptionally low SJT scores may indicate that the candidates have extensive deficiencies in meeting the Foundation Programme person specification, which may raise concerns regarding clinical performance and patient safety. These concerns relate to inability to effectively communicate, prioritise tasks, deal with pressure and demonstrate understanding of the major principles of the GMC’s Good Medical Practice(1) and attributes of a Foundation doctor. The purpose of the review is to decide whether there is clear evidence that the deficiencies indicated by the poor SJT performance either do not exist or are not at a level that would give cause for concern. Candidates who are successful at review are reinstated to the recruitment process. Previous research has shown that the SJT has good predictive validity for supervisor-rated performance in FY trainees.(2) Our objectives are to assess the impact of exceptionally low SJT scores in predicting outcomes relating to clinical performance and training progression during the Foundation Programme.

References

  1. Cousans F, Patterson F, Edwards H, Walker K, McLachlan J, Good D. Evaluating the complementary roles of an SJT and academic assessment for entry into clinical practice. Advances in Health Sciences Education. 2017;22(2):401-413.
  2. Good medical practice. GMC-uk.org. 2021

Doctor Duncan Henderson
duncan.henderson@nhs.scot
UKMEDP150
Understanding the landscape of specific deprivation forms on medical training progression

Approved on 25 June 2021 UKMED Advisory Board

Ongoing Project

The present study aims to explore the role of specific deprivation forms on training progression including: admittance to the medical register, the length of time taken to get onto the medical register, successful completion of foundation training, the length of time taken to complete foundation training and time taken to CCT. The Index of Multiple Deprivation includes seven unequally weighted subdomains including: Health and Disability (13.5%), Education and Training (13.5%) (with subindices for Children and Young people’s deprivation and adult skills deprivation), Crime (9.3%) , Barriers to Housing and Services (9.3%) (including access to amenities, homelessness, household overcrowding), Living Environment (9.3%) (with subindices for air pollution and housing quality), Employment (22.5%) and Income (22.5%) (Figure 1). Each sub-domain and subindex is on a scale of 1 (10% most deprived postcodes) - 10 (10% least deprived postcodes). Studies thus far exploring deprivation in medical education have utilised the overall Index of Multiple Deprivation as opposed to exploring the roles of individual sub-indices. Several studies have already explored the broad questions around the relationships between the overall Index of Multiple Deprivation and admission test performance in UKCAT and Alevel examinations finding the overall Index of Multiple Deprivation to be a predictor of UKCAT and Alevel performance.(1,2) Crude socioeconomic deprivation measures have therefore provided a policy basis for changing medical admissions practices in the UK to include consideration of the overall Index of Multiple Deprivation (3) and have provided a foundation case for medical education widening participation schemes.(4,5) Furthermore, Curtis et al. have provided evidence that gateway courses are proving successful in the undergraduate arena, with many students thriving academically and with the majority graduating as doctors.(7) Studies have not thus far explored the extent to which pre-medical school deprivation, by deprivation form as characterised by IMD subindices, is associated with PG admittance to the medical register including length of time taken to get onto the medical register, successful completion of foundation training, length of time taken to complete foundation training and successful completion of CCT.

References

  1. (1) Sartania N, McClure JD, Sweeting H, Browitt A. Predictive power of UKCAT and other pre-admission measures for performance in a medical school in Glasgow: a cohort study. BMC medical education. 2014 Dec;14(1):1-0.
  2. (2) Tiffin et al. Comparison of the sensitivity of the UKCAT and A levels to sociodemographic characteristics: a national study; BMC Medical Education 2014; 14:7
  3. (3) Fielding et al. ; Do changing medical admissions practices in the UK impact on who is admitted? An interrupted time series analysis; BMJ Open 2018
  4. (4) Blane DN. Medical education in (and for) areas of socio-economic deprivation in the UK.
  5. (5) Tiffin et al.; Widening access to UK medical education for underrepresented socio-economic groups: modelling the impact of the UKCAT in the 2009 cohort; BMJ 2012; 344:e1805
  6. (6) English Indices of Deprivation technical report; https://www.gov.uk/government/publications/english-indices-of-deprivation-2019-technical-report
  7. (7) Curtis, S., Smith, D. A comparison of undergraduate outcomes for students from gateway courses and standard entry medicine courses. BMC Med Educ 20, 4 (2020). https://doi.org/10.1186/s12909-019-1918-y

Doctor Marina Soltan
marinasoltan@gmail.com, M.Soltan@bham.ac.uk, Marina.Soltan@nhs.net
UKMEDP151
Socio-demographic influences on educational choices and outcomes

Approved on 16 December 2021 UKMED Advisory Board

Ongoing Project

Medical applicants from deprived backgrounds have lower odds of gaining an offer, but the odds vary by medical school.[1] Across higher education, students from disadvantaged backgrounds are more likely to ‘undermatch’ - attend less selective courses and institutions given their attainment.[2,3] We previously found that medical applicants 2010 to 2018 from more disadvantaged backgrounds applied to less academically prestigious medical schools closer to home, and had lower odds of an offer (UKMED P89, forthcoming). It is unknown what impact medical school choices, including undermatch, has on subsequent attainment. The impact of the COVID-19 pandemic on applications and outcomes is also uncertain, although young people from poorer backgrounds, including medical applicants,[4] faced additional educational challenges,[5] and there are concerns that teachers awarded more generous A-level grades to more advantaged students.[6] UCL has funded a PhD studentship supervised by Prof Kath Woolf and Dr Gill Wyness, which builds on P89 and similar work in higher education. It aims to examine: 1) The relationship between applicant background and medical school choice, given achievement. 2) The relationship between medical school choice and application success by applicant backgrounds, given academic achievement. 3) The relationship between medical school choice and undergraduate/postgraduate outcomes by applicant background. 4) The impact of the pandemic on medical school choice and outcomes by applicant background, using UCAS data on predicted grades and teacher-assessed grades (results day grades and final grades). Results will be disseminated in a UKMED report and as a PhD thesis. A peer-reviewed publication is anticipated.

References

  1. Steven, K., Dowell, J., Jackson, C. et al. Fair access to medicine? Retrospective analysis of UK medical schools application data 2009-2012 using three measures of socioeconomic status. BMC Med Educ 16, 11 (2016). https://doi.org/10.1186/s12909-016-0536-1
  2. Campbell S., Macmillan, L., Murphy, R., Wyness, G. Matching in the Dark? Inequalities in student to degree match. National Bureau of Economic Research, Working Paper 29215, 2021 DOI 10.3386/w29215
  3. Chowdry, H., Crawford, C., Dearden, L., Goodman, A. and Vignoles, A. (2013), Widening participation in higher education: analysis using linked administrative data. Journal of the Royal Statistical Society: Series A (Statistics in Society), 176: 431-457. https://doi.org/10.1111/j.1467-985X.2012.01043.x
  4. oolf K, Harrison D, McManus CThe attitudes, perceptions and experiences of medical school applicants following the closure of schools and cancellation of public examinations in 2020 due to the COVID-19 pandemic: a cross-sectional questionnaire study of UK medical applicantsBMJ Open 2021;11:e044753. doi: 10.1136/bmjopen-2020-044753
  5. Anders, J., Macmillan, L., Sturgis, P. and Wyness, G. (2021). Inequalities in young peoples’ educational experiences and wellbeing during the Covid-19 pandemic (CEPEO Working Paper No. 21-08). Centre for Education Policy and Equalising Opportunities, UCL.
  6. Richard Murphy & Gill Wyness (2020) Minority report: the impact of predicted grades on university admissions of disadvantaged groups, Education Economics, 28:4, 333-350, DOI: 10.1080/09645292.2020.1761945

Professor Katherine Woolf
k.woolf@ucl.ac.uk
UKMEDP154
How valid are the MSRA, and other selection methods, in predicting in-training performance as part of specialty selection? A longitudinal study using UKMED data to inform COVID-19 contingency plans

Approved on 26 February 2022 UKMED Advisory Board

Ongoing Project

Specialty selection processes have needed to change significantly in response to the COVID-19 pandemic. Prior to January 2021, where face-to-face selection was not feasible, this research proposal was first submitted to seek data as a COVID-19 related emergency to support the medical and dental recruitment and selection (MDRS) subgroups in making informed selection decisions. It is now submitted with more detail, to request permission to disseminate the findings of this research in a peer-reviewed journal. The Multi-Specialty Recruitment Assessment (MSRA) is a computer-based assessment which has been used by several specialties for entry into post-graduate medical training. Given the MSRA can be delivered remotely and has been shown to be a reliable and standardised method of assessment at specialty level, it became an important part of contingency planning for several specialties. Previous research has already demonstrated that the MSRA shows predictive validity in the context of GP (Patterson et al., 2013), and has explored the validity of the MSRA use in other specialties (Patterson et al., 2016; Patterson et al., 2017). Other research has explored the predictive validity of selection processes, in relation to ARCP ratings (Aslet et al., 2020). However, there is a lack of recent research exploring the predictive validity of selection processes across specialties, which look at the use of the MSRA and longitudinal in-training performance outcomes, using ARCP and exam data. This is highlighted in a recent letter published in the PMJ (Ooi & Ooi, 2021). The aim of this research is to explore the predictive validity of selection methods, in specialties which planned to use the MSRA in January 2021, to inform contingency planning. Specifically, the analysis seeks to explore relationships between selection data and in-training performance outcomes, to allow stakeholders to make immediate informed decisions about how the MSRA scores are used, as well as informing longer-term selection approaches. Research will be conducted separately for each specialty: General Practice, Core Psychiatry, Ophthalmology, Obstetrics and Gynaecology, Radiology, Community Sexual and Reproductive Health, Neurosurgery, Anaesthetics, ACCS Emergency Medicine and Paediatrics.

References

  1. A paper exploring the predictive validity of selection for entry into postgraduate training in general practice, using longitudinal studies.
  2. A paper evaluating the recruitment process into UK anaesthesia core training, using a national data linkage study of doctors’ performance at selection and subsequent postgraduate training, with ARCP ratings as outcome measures.
  3. A piece in the PMJ calling for more information about the MSRA and its use across specialties.
  4. A paper evaluating the MSRA for recruitment into radiology specialty training.
  5. A paper exploring the predictive validity of the MSRA and the core medical training selection methods for performance in specialty training.

Professor Fiona Patterson
f.patterson@workpsychologygroup.com
UKMEDP155
The predictive validity of selection for entry into postgraduate training in general practice and psychiatry: what value do face-to-face selection processes add? A retrospective cohort study

Approved on 16 December 2021 UKMED Advisory Board

Ongoing Project

Published in BJPsych Bulletin in March 2024 as "The validity of the selection methods for recruitment to UK core psychiatry training: cohort study"
Medical specialty training selection processes have needed to change substantially in response to the COVID-19 pandemic, where face-to-face selection procedures were not feasible. The Multi-Specialty Recruitment Assessment (MSRA) is a computer-based assessment used by several specialties, including General Practice and Psychiatry. The MSRA has now been used for recruitment into post-graduate medical training for these specialties for a number of years. Given the MSRA has been shown to be a reliable form of standardised assessment, and that it can be delivered remotely it became an important part of the pandemic contingency planning. Previous research has already demonstrated that performance on the MSRA validity predicts future performance in relation to GP training (Patterson et al., 2013; Patterson et al., 2015). While there is no published research into the predictive validity of the MSRA in psychiatric training, unpublished research conducted using Royal College data demonstrated that the MSRA scores are predictive of selection outcomes, postgraduate exam performance and ARCP ratings. In addition, provisional, as yet unpublished findings from analyses of data drawn from the UKMED have also supported the validity of the MSRA in these specialties (application ref UKMEDP154). Bearing in mind the previous evidence supporting the use of the MSRA in these two specialties, the respective selection teams made the decision to use the MSRA as the sole decision maker for selection into General Practice and Psychiatry postgraduate training in 2020 and 2021. The relevant data held in the UKMED were accessed to explore the potential risks around using a single assessment method for selection and to inform immediate decisions. Previously, the value (or otherwise) of face-to-face selection processes were raised in a review of GP selection (Davison et al., 2016). Therefore, the aim of this study is to further explore the incremental value of face-to-face selection processes for recruitment into GP and psychiatry training. Such components of selection generate a relatively high cost to the NHS. The data will be used to explore, in more depth, the relationships between selection assessment performance and subsequent in-training outcomes. This will provide a better understanding of the potential added value of face-to-face selection in this context. This would inform future discussions around use of the MSRA in specialty selection. For example, whether only certain applicants should be prioritised for face-to-face assessment.

References

  1. An evaluation of GP specialty selection which questions the value of face to face selection processes
  2. Research exploring the relationship between general practice selection scores and MRCGP examination performance - it is proposed this application will build upon this research
  3. Research exploring the predictive validity of selection into GP training using longitudinal studies - it is proposed this application will build upon this research

Professor Fiona Patterson
f.patterson@workpsychologygroup.com
UKMEDP157
Leveraging Big Data and AI to Promote Retention of GPs

Approved on 16 December 2021 UKMED Advisory Board

Ongoing Project

The Long Term Plan recognises that ‘the performance of any healthcare system ultimately depends on its people’ yet NHS workforce growth consistently fails to keep pace with demand (NHS, 2019a). To deliver safe and effective care for an ageing population with increasing comorbidities, the NHS requires more doctors who can provide generalist care. To this end, the Interim NHS People Plan (NHS, 2019b) declares the role of the General Practitioner (GP) to be ‘more important than ever’ (p.38) and commits to increasing the number of doctors working in primary care by 6000. Whilst recruitment is one lever to achieve this, this alone will not be enough. Despite an increase in those joining GP training programmes (GMC, 2019), only 72% of newly qualified GPs transition into the substantive workforce yearly and those doing so tend to work just 0.8 FTE. Lead-times for training also mean additional supply will not be seen immediately (NHS, 2019a, 86). With almost a quarter of GPs now aged 55 or over, voluntary early retirement rates trebling over the last decade, and a significant proportion of 35-39 year-olds also leaving the profession (NHS Digital, 2020), there remains a significant long-term risk to GP workforce capacity. This study aims to use big data (from UKMED, HEE’s Trainee Information Systems, ONS, and CQC) and machine learning (ML) techniques to better understand behaviours within postgraduate GP training developing new multi-factorial analyses and predictive models that identity socioeconomic, academic, demographic, and occupational factors associated with in-training events like attrition and extensions to training to extend the evidence base supporting HEE’s strategic objectives (HEE, 2021) and the explicit focus on increasing GP training places, improving GP retention, and increasing the ‘flexibility and responsiveness of medical education and training’ (HEE, 2021:p.10).

References

  1. HEE. (2021) HEE Business Plan 2021/22
  2. GMC. (2019) The state of medical education and practice in the UK. The workforce report.
  3. NHS. (2019a) The NHS Long Term Plan
  4. NHS. (2019b) Interim NHS People Plan
  5. NHS Digital. (2020) General Practice Workforce 30 September 2020, Biannual Regional and Joiners & Leavers tables
  6. Topol, E. (2019) The Topol Review. Preparing the healthcare workforce to deliver the digital future The Topol Review
  7. Government Digital Service. (2020) Data Ethics Framework

Mr Daniel Woolf
daniel.woolf@hee.nhs.uk
UKMEDP158
Higher speciality training in rheumatology: training pathways and outcomes

Approved on 16 December 2021 UKMED Advisory Board

Ongoing Project

The purpose of this study is to describe the rheumatology training pathway and the outcomes of rheumatology training. We will describe the typical duration of higher speciality training in rheumatology, as well as describe the frequency of potential hurdles in rheumatology training pathways which may delay progression i.e. failure of SCE (speciality certificate examinations) or unfavourable outcome at ARCP (annual review of competencies progression. The proportion of trainees taking time out of programme (OOP) during their training will be described, as well as the category of OOP requested (OOP training, experience, research, career break). The proportion of trainees who train less than full-time (LTFT) will be calculated. The proportion of rheumatology trainees successfully completing rheumatology training will be described, as well as outcomes at 12 months’ post-certificate of completion of training (CCT). The intention of this study is to inform future workforce planning within the speciality of rheumatology.

References

  1. Survey of rheumatology trainees in 2011 demonstrating that a significant proportion (around a quarter) were taking higher degrees – it is unclear if this remains the case.
  2. International research (US-based) has demonstrated that non-completion of rheumatology training occurs but that the causes driving this are unclear – this research has never been replicated in the UK and the rate of non-completion here is unknown.
  3. This report by the BSR highlights that there is a current crisis within rheumatology staffing and highlights the importance of careful workforce planning of rheumatology medical trainees.

Doctor Chris Wincup
chris.wincup@nhs.net
UKMEDP159
Post-Foundation training breaks (the 'F3' phenomenon): Evaluating their impact on postgraduate medical training and career progression

Approved on 29 March 2022 UKMED Advisory Board

Ongoing Project

An increasing number of doctors both in the UK and internationally are taking a break along their training pathway. In the UK, the most popular point at which this occurs is after completion of the Foundation Training Programme (FTP) (two years post-graduation) and before commencing a Specialty Training Programme (STP); this has been informally named the ‘Foundation 3 (F3) Year’: a term that is contentious in the postgraduate training community (and will be used cautiously in our research and subsequent reports). This trend has been steadily increasing over the past 10 years to become the most popular post-graduate training break in the UK. Since 2017, over 50% of doctors in each year group undertake an “F3 year”, thereby making this the ‘new normal’. Despite the impact on current and future workforce and healthcare economy, there is little evidence to aid our understanding of this trend. The F3 year has already impacted the fill-rates for core/specialty training posts, which has subsequent effects on healthcare provision through rota-gaps. Longer-term, training breaks effect the doctors’ progression through higher training, thereby delaying completion of training and reducing the number of active Consultants/Attending physicians or qualified General Practitioners/Family Doctors. These issues have financial implications both locally (for Hospital Trusts who need to ensure patient care is provided by a minimum number of staff) and nationally, pertaining to how much of the country’s healthcare budget can be justified for underfilled training posts . The long-term career pathways of doctors who complete an F3 also are not known; and given that a major motivation for the career choice is ‘career specialism uncertainty’, whether this is resolved by the end of F3 is yet to be explored. Our research project aims to explore the medium- and long-term impacts of F3 post-graduate training, regarding career and training progression. The majority of studies published regarding F3 included participants who had not yet completed their Foundation Training or were currently undertaking an F3 year, and therefore explored future career and training intentions, rather than establishing what doctors actually did during their time out of training and the subsequent effects on career and training progression. Our research aims to address this by exploring the journeys of doctors who have taken an F3 year to identify initial motivations for F3, aims/objectives of the year (if any were set) and whether these were met, and whether F3 impacted subsequent career and training trajectories, including any decision to leave medicine. We believe that augmenting UKMED with data from the ‘F3’ survey would add value to the database and bring potential to undertake further analysis around F3s in the future. Access to UKMED would significantly enhance the opportunities for analysis available to us and other researchers. UKMED data on prior educational performance such as Education Performance Measure scores, for example, would enable the research team to see how those taking an F3 previously performed compared to those who do not. UKMED data on outcomes of speciality applications and data on post F3 performance would enable us to look at how they perform on subsequent tests compared royal college membership exams to those who did not take an F3 and whether they progress more quickly on average after F3.how their ARCP outcomes compare to those who did not have a gap in their training. Access to National Training Survey results would enable us to explore measures on burnout in relation to those who undertook an F3 compared to those who did not . Access would also give us access to a greater range of demographic variables to explore, in particular the sociodemographic variables used to identify widening-participation students. The primary research which we are undertaking in the F3 study will generate data that will produce additional insights by being brought together with the UKMED data, for example, motivations for taking F3 from the survey could be related to prior and subsequent self-reported burnout.

References

Doctor Helen Church
Helen.church@nottingham.ac.uk
UKMEDP160
Language and differential GP licencing exam attainment

Approved on 28 April 2022 UKMED Advisory Board

Ongoing Project

As the number of GP training places is expanded, the proportion of international medical graduates (IMGs) has been increasing. Analysis of MRCGP data has shown that examination performance is much worse in IMGs and also in UK graduates of minority ethnic backgrounds. (1,2) Initial suggestions were that these differences may be due to bias arising from racial discrimination in the Clinical Skills Assessment (CSA). (2) However, such differences are also found in the machine marked Applied Knowledge Test (AKT). Indeed, the MRCGP 2019 report states that: “In both the AKT and the CSA, the demographic characteristic which was tied to the biggest difference in performance by candidates on their first attempt was whether or not the candidates had obtained their primary medical qualification in the UK or not”. (3) Furthermore, it has been shown that performance in the AKT predicts performance in the CSA (4) and that CSA examiners do not favour ‘their own kind’. (5) In addition, there is evidence that BAME doctors from UK medical schools are more likely to pass the exams than white doctors from non-UK medical schools. (6) Qualitative research considering differential attainment in the AKT suggests that examination performance may be impeded by differences in undergraduate educational experience, lack of content familiarity and language barriers. (7) These factors have not been sufficiently explored. If confirmed as important confounding factors, training adjustments and examination adaptations, such as the ones already provided for doctors with neural processing difficulties, could be considered to mitigate these.

References

Doctor Victoria Tzortziou Brown
v.tzortzioubrown@qmul.ac.uk
UKMEDP161
Associations between personal and structural factors, and first-attempt membership exam timing and success. Findings from the UK medical education database (UKMED).

Approved on 24 June 2022 UKMED Advisory Board

Ongoing Project

Membership exams are a crucial step in the career progression of a junior doctor. Often a prerequisite for entry into, or advancement along, a training programme, these exams often also represent the first time that a doctor has personal discretion over the timing of an examination. For example, the Part A of the Membership of the Royal College of Surgeons (MRCS) exam can be sat by anyone with a primary medical qualification (e.g., MBBS or MD) from an institution recognised by the General Medical Council (GMC). Yet the MRCS itself is only a formal requirement to enter higher surgical training, which typically starts 4-6 years after the completion of a primary medical degree. With three examination diets a year, in practice, this means that candidates are able to sit any of approximately 12-18 exam sittings; an unusual amount of flexibility for a recent graduate of medical school. Several other factors set membership exams apart from medical school. Notably, the examinations are expensive(Lee et al., 2002), and require a high degree of self-directed learning whilst also working in a clinical role. Furthermore, given the significant cost and time investment that goes with sitting exams, junior doctors are unlikely to apply for an exam before they are relatively certain of a career path. Various other groups have looked at factors that predict success, or performance, at membership examination, with the surgeons’ exam perhaps the most frequently studied(Ellis, Scrimgeour, et al., 2021; Ellis, Brennan, et al., 2021; Scrimgeour et al., 2018; Ellis, Cleland, et al., 2021). However, less is known about the barriers that prevent junior doctors from attempting the MRCS, and relatively little is known about barriers or predictors of success in certain branches of membership examinations, such as occupational medicine or psychiatry. To address this knowledge gap, this study proposes a retrospective cohort analysis of data contained in the UK medical education database to identify personal and structural barriers to attempting, and passing, the first part of the 10 UK membership examinations. In doing so, I hope to identify modifiable barriers to the career progression of junior doctors so that targeted policy and intervention can be devised to support such doctors in achieving their career goals.

References

  1. Lee, T.W.R., Craig, A.R., Ubhi, T. & Reece, A. (2002) “The hidden cost of Membership examinations,” Medical Education, 36(4), pp. 395–395.
  2. Ellis, R., Brennan, P., Scrimgeour, D.S., Lee, A.J. & Cleland, J. (2021) “Performance at medical school selection correlates with success in Part A of the intercollegiate Membership of the Royal College of Surgeons (MRCS) examination,” Postgraduate Medical Journal, p. postgradmedj-2021-139748.
  3. Ellis, R., Cleland, J., Lee, A.J., Scrimgeour, D.S.G. & Brennan, P.A. (2021) A cross-sectional study examining MRCS performance by core surgical training location.
  4. Ellis, R., Scrimgeour, D.S.G., Brennan, P.A., Lee, A.J. & Cleland, J. (2021) “Does performance at medical school predict success at the Intercollegiate Membership of the Royal College of Surgeons (MRCS) examination? A retrospective cohort study,” BMJ Open, 11(8), p. e046615
  5. Scrimgeour, D.S.G., Cleland, J., Lee, A.J. & Brennan, P.A. (2018) “Which factors predict success in the mandatory UK postgraduate surgical exam: The Intercollegiate Membership of the Royal College of Surgeons (MRCS)?,” The Surgeon, 16(4), pp. 220–226.

Doctor Timothy Lindsay
tim.lindsay@me.com
UKMEDP163
Does performance at the Fellowship of the Royal College of Radiology (FRCR) examinations vary according to gender, ethnicity, age, socioeconomic factors, UK medical school and course type?

Approved on 24 June 2022 UKMED Advisory Board

Ongoing Project

In response to the current focus of the focus by the General Medical Council on narrowing the differential attainment gap among trainees of disadvantaged or underrepresented backgrounds, the Royal College of Radiology has recently appointed 4 Leadership Fellows for Fairer Training Outcomes. This data requested is submitted by these 4 fellows to inform the work of the Royal College of Radiology for narrowing the attainment gap. Prior work supported by UKMED has studies the associations of trainee factors with success at the Membership of the Royal College of Surgeons Examinations. They found that there was a significant association between gender, ethnicity, age, graduate status, educational background and socioeconomic status with MRCS pass rates. WE hypothesise that similar group-level differences will be seen for pass rates at the FRCR exams. Understanding these associations can help focus the efforts of the leadership fellowship and guide toward areas of low-hanging fruit for improving success rates among vulnerable trainees.

References

  1. Differential attainment at MRCS according to gender, ethnicity, age and socioeconomic factors: a retrospective cohort study
  2. Does performance at the intercollegiate Membership of the Royal Colleges of Surgeons (MRCS) examination vary according to UK medical school and course type? A retrospective cohort study

Doctor Jina Pakpoor
jina.pakpoor@nhs.net
UKMEDP164
Why don’t all UK orthopaedic trainees complete their training?

Approved on 24 June 2022 UKMED Advisory Board

Ongoing Project

A perfect orthopaedic training programme will identify the best potential consultants, provide training that will maximise that potential and deliver day-one consultants able to provide high-quality care throughout a long career. The current process has limitations: the paucity of outcome measures following CCT makes it difficult to know whether the selection process can be improved and the limited exposure to the specialty prior to application makes it difficult for trainees to know whether they have made the right career choice. Having made the appointments at ST3 we have an obligation to provide universally high-quality training and to support and maximise the potential of all trainees equally. Every trainee who leaves the program represents a failure to meet that obligation, yet little is known about which trainees leave, when they leave and why they leave. Until we have that knowledge it will be difficult to improve the quality of training and to provide training to all demographic groups on an equitable basis. Anecdotal and scientific evidence suggests that some groups may be disadvantaged: female sex and older age have been shown to be associated with a higher likelihood of adverse ARCP outcomes1,2, medical students have articulated their negative impressions of the specialty3, and females who have left surgical training have reported similar negative views4. This data request aims to quantify the rate of attrition from orthopaedic training, to identify groups at greater risk of failing to complete their training, and to form part of a wider study to identify why trainees leave.

References

  1. Differences in progression by surgical specialty: a national cohort study
  2. Personal Characteristics Associated with Progression in Trauma and Orthopaedic Specialty Training: A Longitudinal Cohort Study.
  3. Medical student’s perceptions of and experiences in trauma and orthopaedic surgery: a cross-sectional study.
  4. Why do women leave surgical training? A qualitative and feminist study.

Mr Rob Gregory
rjhgregory@aol.com
UKMEDP165
Variation in quantitative skills of entrants to UK Medical Schools

Approved on 24 June 2022 UKMED Advisory Board

Ongoing Project

This work will build on my previous published work on assessing variation across medical schools and in particular, the statistical learning needs of undergraduate medical students for clinical practice, as informed by a comprehensive survey involving medical graduates. While I have delivered a strong evidence base for introducing more statistical learning into training programmes for medical students in preparation for clinical practice, I am aware from my own teaching experience and from networking across medical schools, that there are barriers to addressing this need, particularly in early years of the undergraduate medical curriculum. A key concern is lack of appropriate prior learning among medical students at entry to medical school. The key aim of this research project is to prepare and publish a clear and accessible synopsis of the mathematical and statistical backgrounds of UK medical students at entry to medical school across the period 2006/7 - 2021. This will prepare the way for taking constructive steps for review of pre-entry level statistical training and foundation year training to ensure that all UK medical graduates acquire adequate core statistical skills in preparation for clinical practice and that this is carried out in a unified way across UK medical schools, including through use of appropriate diagnostic testing. The statistical analysis of the data will be extended to review consistency of performance in the quantitative reasoning component of the UKCAT with participation and performance at secondary school level in quantitative disciplines.

References

  1. MacDougall M. Variation in assessment and standard setting practices across UK undergraduate medicine and the need for a benchmark. International Journal of Medical Education; 6:125-135; doi: 10.5116/ijme.560e.c964
  2. MacDougall M, Cameron HS and Maxwell, SRJ. Medical graduate views on statistical learning needs for clinical practice: a comprehensive survey; 20(1). doi: 10.1186/s12909-019-1842-1

Doctor Margaret MacDougall
Margaret.MacDougall@ed.ac.uk
UKMEDP166
Understanding minority ethnic doctors’ choice of specialty in medicine: the role of academic performance and socioeconomic status

Approved on 25 November 2022 UKMED Advisory Board

Ongoing Project

Despite being well represented in medicine, ethnic minority doctors fall behind their white counterparts in pay (Moberly, 2018) and career progression (Booth, 2021; Rimmer, 2020). This study will analyse the extent to which ‘point of entry factors’ such as socioeconomic status (SES) and education shape the career destinations of ethnic minority doctors. The analysis will expand on previous research (incl. Kumwenda et al., 2018; Smith et al 2015;) in order to extend what is known about the role of SES. For instance, Kumwenda et al (2018) find that having parents with no degree was associated with lower odds of choosing a specialty pathway compared to those whose parents had a degree. We will expand on this and address potential individual-level heterogeneities in allocation into different specialty. Propensity Score Matching will be used to construct samples of ethnic minority individuals comparable to sample of white individuals. We also aim to test whether there are differences in academic performance of ethnic minorities at point of entry, and overtime, how subsequent academic performance differ and their impact on choice of specialty. Kumwenda et al (2017) show that at point of entry, students from independent schools outperform those from state-funded school, but those from state schools are significantly more likely to finish in the highest tier of the educational performance ranking. What we aim to understand is why despite the narrowing and possibly ‘disappearance’ of initial academic disadvantages, ethnic minority students are less likely to allocate into specialty pathways compared with their white counterparts.

References

Doctor Christian Darko
c.k.darko@bham.ac.uk
UKMEDP168
A national study of the moderating influence of gender on postgraduate outcomes in two exemplar specialties (General Practice and Surgery)

Approved on 25 November 2022 UKMED Advisory Board

Ongoing Project

Gender bias is present throughout medical education, with women being less likely to pursue careers in specialties such as surgery and less likely to be appointed to leadership and academic positions (see reference 1 below). Existing research has demonstrated gender bias in education, particularly in clinical environments, and the existence of gendered communities of practice that shape mentorship and support received. This study aims to identify factors that may drive differential attainment and career progression between genders in medical education and training. We aim to develop explanatory models, of increasing complexity, to explore the relationship between prior academic achievement and a number of postgraduate educational and career outcomes in female medical students. Two contrasting exemplar specialities will be used; surgery and general practice, given the differences in gender proportions in each one. We also aim to identify the factors predicting which women are likely to thrive or struggle at a postgraduate level, and use this knowledge to inform the sampling and interviewing approach in the project’s qualitative component, where we will explore how and why any inequalities exist, and develop solutions to these issues. Our findings will be communicated to policymakers and other stakeholders so that potential barriers to female career progression can be addressed.

References

  1. Research exploring student experiences of gender bias within medical education
  2. Research showing women are less likely to pass the MRCS examination (UKMED study)
  3. Research looking at how sociodemographic factors in fluency specialty choice (UKMED study)

Professor Paul Tiffin
paul.tiffin@york.ac.uk
UKMEDP169
Individual and medical school determinants associated with choice of career specialty

Approved on 25 November 2022 UKMED Advisory Board

Ongoing Project

Medical education must adapt to meet the challenges and demands of an ageing population, ensuring that graduates are adequately equipped to look after older adults with complex health and social care needs. There is some limited evidence showing that the quality of undergraduate education in geriatrics increases an individual’s likelihood of becoming a specialist geriatrician, but this is inconclusive. Using the recent British Geriatrics Society third national curriculum survey (2021), and two previous surveys (2008 & 2013), we have an opportunity to further assess whether the type and length of undergraduate teaching in geriatrics influences progression into specialty. We propose to supplement our survey data with UKMED data on recruitment into specialty training, to assess for differences in recruitment between institutions with different educational offerings in geriatrics. Ultimately, our goal is to develop an evidenced-based model for education in geriatrics, that will optimise the medical workforce to serve an ageing population.

References

  1. British Geriatrics Society recommended undergraduate curriculum (2013)
  2. UK survey of undergraduate teaching in ageing and geriatrics 2008
  3. UK survey of undergraduate teaching in ageing and geriatrics 2013
  4. World Health Organization Teaching Geriatrics in Medical Education 1 (2002)
  5. World Health Organization Teaching Geriatrics in Medical Education 2 (2009)

Doctor Grace Pearson
grace.pearson@bristol.ac.uk
UKMEDP170
Retention and progression rates of mature non-graduate students entering medical education

Approved on 25 November 2022 UKMED Advisory Board

Ongoing Project

To increase diversity in the medical school student body, various widening participation policies have been enacted, including the establishment of new Gateway courses. Most of these have been aimed at school leaver groups (outreach, summer schools, Gateway courses). However, these entry tariffs are still considerable and potential applicants who left school with non-standard qualifications (e.g., BTECs), largely cannot access either the standard entry or the Gateway route. Instead, they gain suitable Level 3 qualifications, by attending eligible FE Colleges to gain Access to HE (medicine) diplomas (although not all Gateway or standard entry courses will accept these) or take A-levels as independent candidates. These mature candidates face multiple, significant barriers in applying for medicine, and not all of these disappear after they gain entry to the course. While a limited number of studies compare WP students’ progression (either Gateway graduates or standard entry WP students) to the rest of the student cohort (1, 2), there are no studies that specifically look at mature, non-graduate students, defined as applicants over 21 years old entering the course with qualifications no higher than Level 3. We want to understand the retention and progression of these students throughout the medical school, and into the start of their medical careers. We feel it is important to research this previously neglected group of medical students. Although mature non-graduate applicants may have very different pathways to medicine, they are likely to belong to some of the most disadvantaged and under-represented groups accessing Higher Education, and even more so for medicine. It would be interesting to compare their socio-demographic profile to other groups of applicants. A null hypothesis that progression and attainment of the mature, non-graduate student group is not statistically different from other students, would inform whether widening access efforts to this group are justified. Practically, if the null hypothesis is correct, this might lead to changes in admissions, such as increased acceptance of Access to HE (Medicine) qualifications for entry into more courses. Thus, the findings would have clear implications for the way the selection is implemented within undergraduate medicine.

References

  1. 1. Curtis, S., Smith, D. (2020). A comparison of undergraduate outcomes for students from gateway courses and standard entry medicine courses. BMC Med Educ 20, 4.
  2. 2. Woolf, K., Potts, H.W. W and McManus, I.C., (2011). Ethnicity and academic performance in UK trained doctors and medical students: systematic review and meta-analysis. BMJ, 342: d901.

Doctor Nana Sartania
Nana.Sartania@glasgow.ac.uk
UKMEDP172
What sociodemographic and academic factors are associated with success in application to the Specialised Foundation Programme? A retrospective study of the UKMED Database

Approved on 25 November 2022 UKMED Advisory Board

Ongoing Project

The UK Specialised Foundation Programme (SFP) facilitates foundation doctors to develop research, teaching, and leadership skills. Although growing in popularity, only 5% of junior doctor posts are academic, resulting in great competition for posts (1). There is a lack of published data on characteristics of individuals who are successful in application to the SFP. Only one study, published by Donaldson et al., showed that medical schools with an enforced intercalation period had a greater proportion of applicants succeeding in applying for the SFP (2). However, they didn’t account for confounding factors in their analyses. The British Medical Association is concerned about dwindling numbers of medical academics (3). Since 2010/ 11, medical student intake has increased by 31% whereas the medical teaching workforce has increased by 0.4% (3). There is therefore a clear imperative to increase medical academic numbers. An appraisal of barriers to entry and predictors of success is therefore necessary. One of the major components of an SFP application is that one has evidence of academic outputs (publications, presentations, prizes, or additional degrees). Often, these achievements require significant work outside of the regular medical school timetable, and so we hypothesise that students who are required to self-fund their degrees or associated living expenses, are less likely to generate academic outputs due to lack of time. Understanding the relationships between sociodemographic status and likelihood of success could highlight that changes are needed within medical schools to support students with certain demographic characteristics into SFPs.

References

Doctor Susan Ball
S.Ball3@exeter.ac.uk
UKMEDP174
Does performance at post-graduate surgical membership examination vary according to different types of anatomy teaching in undergraduate medical education?

Approved on 25 November 2022 UKMED Advisory Board

Ongoing Project

In the UK, there are different approaches to teaching anatomy to future doctors. Cadaveric dissection and prosection are the most traditional and dominant approaches in anatomy education for undergraduate medical training. (1) In recent years, alternative teaching methods, such as problem-based anatomy learning cases and technological innovations such as three-dimensional (3D) anatomy models are adopted in anatomy education. Despite their own benefits and limitations, acquiring an adequate level of anatomy knowledge remains an essential criterion for performing well in postgraduate surgical training. (2),(3) There are only a limited number of studies investigating the correlation between undergraduate anatomy teaching and performance in postgraduate surgical exams. This study will be the first quantitative study to explore the relationship between the mode of undergraduate anatomy education and postgraduate surgical training performance. In this study, we will use the MRCS first-attempt passing rate as our quantitative measurement of postgraduate surgical training performance. The MRCS is a well-respected examination which provides a good indicative marker in clinical anatomy competence. Overall, this study aims to identify the correlation between the mode of undergraduate anatomy teaching and performance in postgraduate surgical exams. Through this research, we hope to determine the impact of undergraduate anatomy education on the anatomical competency of the graduates, which is translatable to the performance of providing surgical service to our patients. We wish to provide educational implications on the undergraduate anatomy curricula in a national context and provide insight for future improvement in higher surgical training performance.

References

  1. Selcuk İ, Tatar I, Huri E. Cadaveric anatomy and dissection in surgical training. Turk J Obstet Gynecol. 2019;16(1):72-5.
  2. Williams SR, Thompson KL, Notebaert AJ, Sinning AR. Prosection or Dissection: Which is Best for Teaching the Anatomy of the Hand and Foot? Anat Sci Educ. 2019;12(2):173-80.
  3. Ghazanfar H, Rashid S, Hussain A, Ghazanfar M, Ghazanfar A, Javaid A. Cadaveric Dissection a Thing of the Past? The Insight of Consultants, Fellows, and Residents. Cureus. 2018;10(4):e2418.

Doctor Gary Tse
Gary.tse@kmms.ac.uk
UKMEDP178
Understanding determinants of education, including specific deprivation forms, on physician medical training progression

Approved on 25 November 2022 UKMED Advisory Board

Ongoing Project

The present study aims to explore the role of specific deprivation forms on Physician training progression including: admittance to the medical register, the length of time taken to get onto the medical register, successful completion of foundation training, the length of time taken to complete foundation training, performance at critical progression points, and time taken to CCT. This builds upon a previous UKMED application exploring the same in GP training finding specific pre-medical school IMD deprivation forms as independent predictors at various progression points. The Index of Multiple Deprivation includes seven unequally weighted subdomains including: Health and Disability (13.5%), Education and Training (13.5%) (with subindices for Children and Young people’s deprivation and adult skills deprivation), Crime (9.3%) , Barriers to Housing and Services (9.3%) (including access to amenities, homelessness, household overcrowding), Living Environment (9.3%) (with subindices for air pollution and housing quality), Employment (22.5%) and Income (22.5%) (Figure 1). Each sub-domain and subindex is on a scale of 1 (10% most deprived postcodes) - 10 (10% least deprived postcodes). (1) Equal access to education is among the basic human rights to which all are entitled (2). Yet, several studies have shown staggering educational gaps between various groups suggestive of educational inequalities. Within medical education, published works have reported educational inequality gaps with respect to: the overall Index of Multiple Deprivation (IMD) and admission to medical school (3), ethnicity with differential attainment in the UKCAT (4) and ethnicity and overall IMD with performance in the MRCS (5). Two published studies have explored predictors of examination success in MRCS (Membership of the Royal College of Surgeons) for surgical speciality training and MRCP(UK) for physician speciality training; both training pathways are typically a minimum of seven years following foundation training. The former study considered EPM, SJT, gender and ethnicity finding EPM to be the most significant measure of predicting MRCS success (6). The later study considered age, UCAS tariff and UKCAT subsection performance finding that the abilities assessed by aptitude and skills and verbal reasoning may be the most important cognitive attributes of those routinely assessed at selection for predicting future clinical performance in MRCP (UK) (7). The former have sparked a body of literature influencing policy making around the implementation of widening participation schemes. Several studies have already explored the broad questions around the relationships between the overall Index of Multiple Deprivation and admission test performance in UKCAT and A-level examinations finding the overall Index of Multiple Deprivation to be a predictor of UKCAT and A-level performance. (8,9) Crude socioeconomic deprivation measures have therefore provided a policy basis for changing medical admissions practices in the UK to include consideration of the overall Index of Multiple Deprivation (10) and have provided a foundation case for medical education widening participation schemes.(11,12) Furthermore, Curtis et al. have provided evidence that gateway courses are proving successful in the undergraduate arena, with many students thriving academically and with the majority graduating as doctors.(13) Studies thus far have not explored the multiple lenses of inequality in medical education such as demographic, protected characteristics and deprivation forms including the roles of the seven IMD domains and further granular subdomains with indicators for specific relevant social determinants of education, in isolation or intersectionally, as modulators of progression among Physicians. Furthermore, studies have not considered educational inequalities in medical education as a longitudinal phenomenon from pre-undergraduate education, often childhood, until CCT nor monitoring metrics.

References

  1. (1) English Indices of Deprivation technical report; https://www.gov.uk/government/publications/english-indices-of-deprivation-2019-technical-report
  2. (2) Legislation. Human Rights Act Legislation. 1998.
  3. (3) al. Te. Comparison of the sensitivity of the UKCAT and A levels to sociodemographic characteristics: a national study. BMC Medical Education. 2014 14; 7.
  4. (4) Woolf K. Taking the difference out of attainment. BMJ. 2020;: 368.
  5. (5) Ellis, R. et al.. Differential attainment at MRCS according to gender, ethnicity, age and socioeconomic factors: a retrospective cohort study. Journal of the Royal Society of Medicine. 2022.
  6. (6) Ellis R, Scrimgeour D, Brennan P, Lee A, Cleland J. Does performance at medical school predict success at the Intercollegiate Membership of the Royal Colleege of Surgeons (MRCS) examination? A retrospective cohort study. BMJ Open. 2020.
  7. (7) Paton L, Mcmanus I, Cheung K, Smith D, Tiffin P. Can achievement of medical admissions tests predict future performance in postgraduate clincial assessments? A UK based national cohort study. BMJ Open. 2021;
  8. (8) Sartania N, McClure JD, Sweeting H, Browitt A. Predictive power of UKCAT and other pre-admission measures for performance in a medical school in Glasgow: a cohort study. BMC medical education. 2014 Dec;14(1):1-0.
  9. (9) Tiffin et al. Comparison of the sensitivity of the UKCAT and A levels to sociodemographic characteristics: a national study; BMC Medical Education 2014; 14:7
  10. (10) Fielding et al. ; Do changing medical admissions practices in the UK impact on who is admitted? An interrupted time series analysis; BMJ Open 2018
  11. (11) Blane DN. Medical education in (and for) areas of socio-economic deprivation in the UK.
  12. (12) Tiffin et al.; Widening access to UK medical education for underrepresented socio-economic groups: modelling the impact of the UKCAT in the 2009 cohort; BMJ 2012; 344:e1805
  13. (13) Curtis, S., Smith, D. A comparison of undergraduate outcomes for students from gateway courses and standard entry medicine courses. BMC Med Educ 20, 4 (2020). https://doi.org/10.1186/s12909-019-1918-y

Doctor Marina Soltan
marinasoltan@gmail.com
UKMEDP180
Diversity in Surgery – Trends in Core Surgical Training

Approved on 20 February 2023 UKMED Advisory Board

Ongoing Project

Interest in Equality, diversity and inclusivity (ED&I) in medicine and surgery is increasing. It has been shown that diversity can help organizations improve patient care quality[1]. The Royal Colleges of Surgeons within the UK have been making a concerted effort to focus on diversity. In 2021 Baroness Kennedy was commissioned by the newly elected President of Royal College of Surgeons England (RCSEng) to report on diversity within the college, and experiences of minorities. Similarly, the Royal College of Surgeons Edinburgh (RCSEd) President recently has stated that “Equality and diversity is not just a commitment— it is about living and breathing inclusion of all our people”. There has been interest in improving women’s representation within UK surgical specialities. Two initiatives of note include the Women in Surgery Network and Emerging Leaders Fellowship. Newman et al. [2] note that there has been an improvement in gender diversity within some UK surgical specialties, however there is need for improvement in others. Their work however, only focuses on gender diversity and does not take into account other demographics such as race, ethnicity or sexuality. Kim et al. [3] found that racial/ethnic minorities make up a significantly lower proportion of surgical residents in the USA than that of the general population. They looked into what they describe as the “leak in the diversity pipeline”; an attempt to find at what stage in surgical training cohorts start to become less diverse. They found that although there have been improvements in the diversity of medical school admission, these improvements are lost when doctors enter surgical residency programmes. There has been research into demographic predictors for success at the Membership of Royal College of Surgeons (MRCS) examinations. Ellis and colleagues found that males were more likely to pass MRCS part A on first attempt, and black and minority ethnic groups were significantly less likely to pass MRCS Part B at their first attempt B [4]. CST is the first stage of surgical training in the UK. Doctors are invited to apply for CST once they have completed their foundation programme. As this is the first step of most surgical careers within the UK, it would stand that if equality and diversity is not achieved at this important step, then surgical profession as a whole will be far less likely to achieve their equality and diversity goals.

References

  1. Gomez LE, Bernet P. Diversity improves performance and outcomes. J Natl Med Assoc [Internet]. 2019;111(4):383–92. Available from: https://doi.org/10.1016/j.jnma.2019.01.006
  2. Newman TH, Parry MG, Zakeri R, Pegna V, Nagle A, Bhatti F, et al. Gender diversity in UK surgical specialties: A national observational study. BMJ Open. 2022;12(2):1–8.
  3. Kim Y, Kassam AF, McElroy IE, Lee S, Tanious A, Chou EL, et al. The current status of the diversity pipeline in surgical training. Am J Surg [Internet]. 2021;(September 2021). Available from: https://doi.org/10.1016/j.amjsurg.2021.11.006
  4. Ellis R, Brennan PA, Lee AJ, Scrimgeour DS, Cleland J. Differential attainment at MRCS according to gender, ethnicity, age and socioeconomic factors: a retrospective cohort study. Journal of the Royal Society of Medicine. 2022;115(7):257-272. doi:10.1177/01410768221079018

Mr Matthew Newman
matthew.newman2@nhs.scot
UKMEDP181
Modelling the relationship between University Clinical Aptitude Test (UCAT) scores and postgraduate performance and specialty choice

Approved on 30 June 2023 UKMED Advisory Board

Ongoing Project

Background One of the original aims of introducing the UCAT was to help selectors choose individuals most likely to succeed in a clinical career. Performance in “high-fidelity” postgraduate clinical exams are an accessible proxy for actual practice and can be predicted from UCAT scores (Paton, McManus et al. 2022). The predictive validity of the situational judgement test (SJT) component of the UCAT can now be evaluated in this respect. The SJT scores may be especially relevant to clinical examination performance as they test knowledge of interpersonal effectiveness. Given the workforce shortages being experienced by key specialties it will also be important to consider a novel “pareto-optimal” approach. This models optimal trade-offs between postgraduate pass and recruitment rates, as well as overall fill rates for specialty training. Methods and Approach A series of univariable, multivariable and, ultimately, path models will be developed and tested. These will evaluate the hypothesised causal relationships between educational and selection assessment scores (including the UCAT) and eventual performance in postgraduate clinical examinations. Predictors of specialty choice will also be modelled using regression-based approaches. ‘Efficiency frontiers’ will be generated for pareto-optimal models that indicate optimal combinations of trade-offs (De Corte, Sackett et al. 2011). Numerical simulation will be used for workforce modelling for general practice, under various conditions, as an exemplar speciality experiencing acute workforce shortages. Our findings will be crucial to understanding the role of the UCAT in relation to “reverse engineering” medical selection. This maps selection to workforce requirements for effective healthcare delivery.

References

  1. Paton, L. W., I. C. McManus, K. Y. F. Cheung, D. T. Smith and P. A. Tiffin (2022). "Can achievement at medical admission tests predict future performance in postgraduate clinical assessments? A UK-based national cohort study." BMJ Open 12(2): e056129.
  2. De Corte, W., P. Sackett and F. Lievens (2011). "Designing Pareto-Optimal Selection Systems: Formalizing the Decisions Required for Selection System Development." Journal of Applied Psychology 96: 907-926.

Doctor Taha Khan
taha.khan6@nhs.net
UKMEDP182
Ophthalmology Educational Performance and Treatment Intervention Complications: the OPTIC Study

Approved on 30 June 2023 UKMED Advisory Board

Ongoing Project

Educational factors, including assessment standards, should be linked to the quality of patient care. There is a small number of US-based examples where medical regulatory factors have been were shown to predict clinical outcomes [2,3,4]. In the UK there are virtually no examples where patient educational outcomes have been linked, at scale, to the quality of patient care. By linking data from the National Ophthalmology Database (NOD) Cataract audit to the UKMED we can create a proof of concept for this type of research in the UK. There is some knowledge of the patient-level risk factors for complications related to cataract surgery [1]. However, there is no existing evidence relating to the physician related predictors of complications in this context beyond the association between increasing experience and lower complication rates, which is well attested. In order to complete a Certificate of Completion of Training (CCT), trainee ophthalmologists in training must pass the Fellowship examinations of the Royal College of Ophthalmologists (FMRCOphth). This consists of: Part 1 FRCOphth examination (2 x 2 hour MCQ papers)- assessing understanding of patient investigations and relevant basic and clinical sciences; Refraction Certificate (10 station OSCE)- covering optical devices and refraction; Part 2 FRCOphth examination (2 x MCQ papers and an oral exam involving a structured viva and clinical OSCE). By linking NOD and UKMED data at the level of the individual doctor we can model the relationship between surgical performance and academic performance in the different components of the FRCOphth. This means we can gain an understanding of how performance in the FRCOphth may be associated with patient outcomes for cataract surgery. This may help the Royal College consider any implications for standard setting for the FRCOphth. For example, the maximum number of sittings as the clinical component that should be permitted. It will also create knowledge about how clinical outcomes for ophthalmologists may be related, or unrelated, to world region of primary medical qualification.

References

  1. 1. Greenberg, P. B., V. L. Tseng, W. C. Wu, J. Liu, L. Jiang, C. K. Chen, I. U. Scott and P. D. Friedmann (2011). "Prevalence and predictors of ocular complications associated with cataract surgery in United States veterans." Ophthalmology 118(3): 507-514.
  2. 2. Norcini, J. J., J. R. Boulet, W. D. Dauphinee, A. Opalek, I. D. Krantz and S. T. Anderson (2010). "Evaluating the quality of care provided by graduates of international medical schools." Health Affairs 29(8): 1461-1468.
  3. 3. Norcini, J. J., J. R. Boulet, A. Opalek and W. D. Dauphinee (2014). "The relationship between licensing examination performance and the outcomes of care by international medical school graduates." Acad Med 89(8): 1157-1162.
  4. 4. Norcini, J. J., J. R. Boulet, A. Opalek and W. D. Dauphinee (2018). "Specialty Board Certification Rate as an Outcome Metric for GME Training Institutions: A Relationship With Quality of Care." Evaluation & the Health Professions 43(3): 143-148.

Professor Paul Tiffin
paul.tiffin@york.ac.uk
UKMEDP184
Effects of Context Familiarity on Ethnic Subgroup Differences and Predictive Validity in the UCAT SJT.

Approved on 16 August 2023 UKMED Advisory Board

Ongoing Project

In the field of assessment and selection, there is an urgent need to address ethnic sub-group differences in attainment, which have persisted over many years. To date, there has been very little research exploring causality to explain why such differences exist, and thereby design appropriate mitigation strategies to minimise these differences. In designing selection methods, the importance of context is recognised, as evidenced by “contextualised” selection procedures. One key overlooked issue is the extent to which the context depicted in a selection procedure is (un)familiar to applicants, which in turn may impact ethnic sub-groups differently. To address this issue, selection researchers often opt for a less job-related context when designing selection procedures, which is assumed to be familiar to all applicants (i.e., setting items in an educational context) as an alternative contextualisation. However, there is no evidence that such an alternative contextualisation is indeed effective in terms of reducing ethnic subgroup differences, while at the same time ensuring validity. This research aims to further our understanding of contextualisation decisions in designing selection procedures, specifically SJTs, investigating its effects not only on predictive validity but also on ethnic subgroup differences. The research will utilise the 2013 cohort of applicants that completed UCAT. This will allow the predictive validity data collected in 2015-16 to be included. Phase 1 of the research will include the sub-sample that took part in the predictive validity research. Phase 2 will then seek to replicate findings on the full 2013 cohort.

References

  1. In this paper, we examined ethnic sub-group differences related to the UKCAT 2012 and linked this to SES. So, we have experience with UKCAT data.
  2. In this paper, we explored the extent to which the UCAT SJT was able to predict performance on the undergraduate medical and dental school programs at 4 universities.
  3. This conceptual paper puts forward key arguments as to why contextualization is a relevant factor in research that aims to lower ethnic sub-group differences.
  4. This research was another example (just like this project) of attempts to reduce ethnic sub-group differences. Specifically, we discovered that adopting an open response format in SJTs lowered ethnic sub-group differences. In this project, we aim to investigate the potential beneficial role of contextualization.
  5. This pioneering research was the first ever large-scale examination of the use of SJTs in medical education admission contexts (Flemisch admission exam). Over the years, it has therefore received a lot of citations and generated much impact.

Professor Fiona Patterson
f.patterson@workpsychologygroup.com
UKMEDP188
Is the pipeline for early career clinical academics functioning? An analysis of what determines participation in the Specialised Foundation Programme and beyond with the UKMED.

Approved on 30 June 2023 UKMED Advisory Board

Ongoing Project

Clinical academics comprise a small but important sector of the medical workforce (1,2). A structured career pipeline exists, with an early step being the Specialised Foundation Programme (SFP, formerly Academic Foundation Programme [AFP]) which provides opportunities for newly qualified doctors to develop foundational research skills, which can then be applied to further clinical academic training (3). For some, SFP is their first experience of research while for others (i.e., those with intercalated or prior degrees), they will build upon prior experience. However, there has been a decline in the proportion of clinical academics and there are growing concerns that this pipeline is ‘leaky’, with doctors exiting this career path early (1,2,4). Barriers to engagement with academia present themselves before, during and after undergraduate medical training, yielding attrition. This disproportionately affects certain groups, such as women and those from socioeconomically disadvantaged, or minority ethnic backgrounds (2,4). Identifying these points of attrition is crucial for maintaining a sustainable pipeline of diverse clinical academics (1,2). Work suggests students from ‘research active’ institutions are more likely to apply to the SFP (5), but there are no data on the characteristics of who applies to and enters the SFP, nor who then pursues further clinical academic training (5). We will investigate the characteristics of those who pursue clinical academic careers – primarily, the effects of socioeconomic status, academic attainment, and demographic characteristics (i.e. gender and ethnicity), on medical students choosing to apply to, and then enter the SFP, and then choose to pursue further academic training.

References

  1. The ‘Clinical academics in the NHS inquiry’ found clinical research in the NHS is ‘under threat’ and ‘in decline’ and that the careers for clinical academics are ‘part of decline’, with fewer young clinical academics entering the workforce in the next decade.
  2. This report shows the importance of protected clinical academic jobs and highlights the ‘leaky pipeline’ and where barriers disproportionately affected certain groups with specific protected characteristics (such as gender and ethnicity).
  3. This paper highlights the structure of the SFP/AFP, the diverse range of skills gained within research, leadership and education by those on the programme and areas for potential future development of the programme.
  4. This report highlights a decline in the proportion of clinical academics (particularly at the senior lecturer level) since its peak in 2010 and highlights that women are less likely to occupy senior academic positions.
  5. A search of the ‘PubMed’ database for ‘(("Academic Foundation Programme") OR ("Specialised Foundation Programme")) AND (Medical Education)’ at present (12/05/2023) reveals only one peer-reviewed study which analyses national data relating to the SFP (Donaldson et, al.) and highlights the lack of research around this area.

Doctor Robert Bain
r.bain2@newcastle.ac.uk
UKMEDP190
GPs educational trajectories: Inequalities in access to medical degrees and determinants of academic performance

Approved on 30 June 2023 UKMED Advisory Board

Ongoing Project

There is substantial need for analysing what are the causes and consequences of inequality in access to medical degrees. Especially because there remains a lack of understanding of the educational and professional paths of medical students and postgraduates pursuing primary care training. Previous research has shown that students from more privileged backgrounds and independent schools are more likely to attend medical school, which may have implications for transitions to general practices and the healthcare sector. Furthermore, the General Medical Council has raised concerns about the lack of diversity in the medical workforce and its potential negative impact on patients in economically deprived areas. While existing evidence on access to medical school and its outcomes is mostly correlational, it is necessary to assess undergraduates' career intentions to later understand how this affects the transition to the labour market, particularly regarding general practice. Currently, only a small percentage of graduates express interest in becoming GPs, mostly female. Exploring the educational trajectories is necessary to make the health policies to retain, increase diversity, and attract more students to general practice more effective.

References

  1. Study 1 – Strømme and Hansen (2017): Closure in the elite professions: the field of law and medicine in an egalitarian context - Sheds light on the challenges of ensuring equitable access to elite professions and the mechanisms that maintain the exclusivity of these professions - By examining the role of socio-economics background and institutional strategies, the study provides valuable insights into how intergenerational closure can be perpetuated even in a context of educational expansion - This information can inform efforts to increase access to elite professions and reduce social inequalities The paper by Strømme and Hansen (2017) is relevant to the proposed study as it explores the phenomenon of intergenerational closure in the elite professions of law and medicine, and investigates the extent to which these professions disproportionately recruit students from socio-economically advantaged backgrounds.
  2. Study 2 – Nussbaum et al, 2021: Inequalities in the distribution of the general practice workforce in England: a practice-level longitudinal analysis longitudinal analysis. - Significant workforce inequalities exist and are even increasing for several key general practice roles, with workforce shortages disproportionately affecting more deprived areas - Policy solutions are urgently needed to ensure an equitably distributed workforce and reduce health inequities. The paper by Nussbaum et al. (2021) is relevant to the proposed study as it highlights the significant workforce inequalities that exist in the general practice workforce in England and the need for policy solutions to ensure an equitably distributed workforce and reduce health inequities.
  3. Study 3 – Asaria et al., 2015: Unequal socioeconomic distribution of the primary care workforce: whole-population small area longitudinal study - Introduces a new small area level method for measuring inequality in general practitioner supply that focuses specifically on socioeconomic inequality and captures inequality within National Health Service (NHS) administrative areas as well as between them - Notes that it is not known how much more need for primary care there is in deprived areas relative to affluent areas The paper by Asaria et al. (2015) is relevant to the proposed study as it introduces a new method for measuring inequality in general practitioner supply that focuses on socioeconomic inequality and highlights the lack of understanding of the differences in primary care needs between deprived and affluent areas.
  4. Study 4 – Barratt, 2017: Dying in hospital: socioeconomic inequality trends in England - There was substantial ‘pro-rich’ inequality, with an estimated difference of 5.95 percentage points in the proportion of people dying in hospital (confidence interval 5.26 to 6.63), comparing the most and least deprived neighbourhoods in 2011/2012 - Highlights that greater understanding of the reasons for such inequality is required before policy changes can be determined The paper by Barratt (2017) is relevant to the proposed study as it highlights substantial socioeconomic inequality in the proportion of people dying in hospitals in England, emphasising the need for greater understanding of the reasons for such inequality before determining policy changes.

Doctor Catia Nicodemo
catia.nicodemo@economics.ox.ac.uk
UKMEDP191
Statistical analysis of the recruitment of doctors, as a result of changing the recruitment processes

Approved on 30 June 2023 UKMED Advisory Board

Ongoing Project

The COVID-19 pandemic has instigated substantial changes in medical and dental specialty training selection processes, transitioning from face-to-face methods to digital approaches. Our study aims to assess the fairness and equity of digital recruitment methods implemented by Health Education England (HEE) during the pandemic, using data from the UK Medical Education Database (UKMED). The primary objectives of this study are to identify inequalities in accessing health careers for underrepresented groups, evaluate the effectiveness of digital recruitment in comparison to face-to-face methods, and assess the reliability and validity of the selection process. This research will provide a critical understanding of the implications of digital selection methods on applicant demographics and subsequent training outcomes. We will utilize logistic regression models to investigate the influence of demographic factors on recruitment outcomes, apply standardised recruitment ratios to measure inequalities, and use interrupted time series (ITS) analysis to assess the impact of digital recruitment methods compared to traditional approaches. This methodological approach will enable us to account for potential confounding factors and systematically compare the effectiveness of different recruitment strategies. The findings of this study will contribute to the development of a robust evidence base to guide HEE's recruitment strategies, ensuring fair and equitable processes for all applicants. Furthermore, it will inform discussions on the potential value of face-to-face selection processes and the implications for selection in undergraduate medicine. By providing evidence on the effects of digital recruitment methods, this research will facilitate evidence-based decision-making in medical and dental training selection processes.

References

  1. Bhatti, N., & Nayar, V. (2023). Performance of ethnic minority versus White doctors in MRCGP assessment. British Journal of General Practice, 73(730), 202–202. Authors review the performance of ethnic minority versus White doctors in MRCGP assessment (2016–2021), a similar approach to that taken in the proposed study but considers additional protected characteristics.
  2. Challinor, A. and Whyler, J. (2022), "The impact of the changes to United Kingdom psychiatry training recruitment in 2021", The Journal of Mental Health Training, Education and Practice, Vol. 17 No. 4, pp. 335-341. The purpose of this paper was to review and critically evaluate UK psychiatry national recruitment process for 2021, which was re-structured following the COVID-19 pandemic. This project aims to build on this by exploring recruitment process changes for all medical specialties.
  3. Tridente, A., Parry-Jones, J., Chandrashekaraiah, S. et al. Differential attainment and recruitment to Intensive Care Medicine Training in the UK, 2018–2020. BMC Med Educ 22, 672 (2022). The purpose of this study is to establish whether any specific individual characteristics are associated with performance in selection for entry into specialty training in Intensive Care in the United Kingdom. Similar to previous citation, this study aims to build on this by exploring the effect of individual characteristics on performance and recruitment for all medical specialties.

Mr Steve Hodgson
Steve.Hodgson@rsmuk.com
UKMEDP192
Investigating the effect of specific widening participation initiatives on medical school offers and admissions

Approved on 30 June 2023 UKMED Advisory Board

Ongoing Project

Students from non-traditional backgrounds, including lower socio-economic backgrounds, are under-represented in UK medical schools (1-4). Nationwide, widening participation initiatives driven by charities, government bodies and higher education institutions themselves have attempted to address these disparities and support under-represented students in their applications to medical school. However, the impact of such interventions are not always easy to evaluate. They typically lack suitable control groups, and it can be challenging to track students and their applications to multiple medical schools, which can occur over a period of years. We intend to determine the outcomes for students who apply to medical schools having engaged with one of two major national widening participation programmes; the Medical School Council-sponsored summer schools, and the Sutton Trust-sponsored ‘Pathways to Medicine’ programme. We will compare them to control groups taken from UKMED. We will compare the numbers of these students who apply who receive offers, the average number of offers per student, and the numbers who commence studying medicine. For those students involved in earlier iterations of these programmes, we also wish to determine how many graduate successfully. This work will determine whether these specific programmes increase the chances of participants accessing medicine, and provide wider insight into the utility, design and provision of such widening participation initiatives.

References

  1. Medical Schools Council. Selection Alliance 2019 Report.
  2. Garrud P. Help and hindrance in widening participation: commissioned research report. 2014.
  3. Smith T, Noble M, Noble S, et al. English indices of deprivation 2015: technical report.2015.
  4. ) Steven K, Dowell J, Jackson C, et al. Fair access to medicine? Retrospective analysis of UK medical schools application data 2009-2012 using three measures of socioeconomic status. BMC Med Educ. 2016;16(1):1–10.

Professor Kevin Murphy
k.g.murphy@imperial.ac.uk
UKMEDP194
Exploring the postgraduate attainment gap between graduates from different course types

Approved on 30 June 2023 UKMED Advisory Board

Ongoing Project

There is a drive to widen the demographic representation of doctors in the UK, so they more closely represent the populations they serve. Increasing representation from lower socioeconomic groups is a UK government and Medical Schools Council priority. One way this is being achieved is through six-year “gateway” courses. Gateway students (also known as widening participation students) enter medical school with lower academic grades in acknowledgement of social and educational disadvantages experienced prior to university. Although these courses are successful in narrowing the attainment gap, a significant gap remains in undergraduate outcomes (Curtis & Smith 2020). A recent study provides evidence for the continuation of this attainment gap in postgraduate medical education. Gateway graduates are less likely to pass their first attempt at any membership exam than graduates of standard entry medicine courses (39% vs 63%) (Elmansouri, Curtis, Nursaw & Smith 2023). Gateway graduates are also less likely to be offered a level 1 training position on their first application (75% vs 82%). This proposed study would aim to use data from UKMED to undertake an expanded and detailed analysis of the postgraduate outcomes of all doctors who graduated from UK gateway courses, including the different elements of membership exams compared to graduates of traditional courses. The outcomes of this study would be used to inform a wider qualitative study to explore the trainees experience of the attainment gap.

References

  1. Curtis, S., Smith, D. A comparison of undergraduate outcomes for students from gateway courses and standard entry medicine courses. BMC Med Educ 20, 4 (2020).
  2. Elmansouri, A., Curtis, S., Nursaw, C. et al. How do the post-graduation outcomes of students from gateway courses compare to those from standard entry medicine courses at the same medical schools?. BMC Med Educ 23, 298 (2023).

Professor Sally Curtis
sac3@soton.ac.uk
UKMEDP196
Change in the demographic profile of medicine applicants and entrants since Selecting for Excellence

Approved on 01 December 2023 UKMED Advisory Board

Ongoing Project

In 2014, Medical Schools Council published the final report of a 2-year project, Selecting for Excellence (SfE), that set out the substantial challenges facing the sector in widening access and widening participation in medicine. That report included statistical summaries of the profile of medical students in terms of gender (55% female), ethnicity (62% white), social class (87% higher managerial & professional), and POLAR (5% from least advantaged quintile). Commissioned research (Garrud, 2014) also showed that around 80% of applicants came from just 20% of secondary schools/colleges and 50% of secondary schools/colleges contributed no applicants at all. The ensuing decade has seen a remarkable effort by medical schools to widen access and participation (e.g. MSCSA Annual Report, 2019) and, after ten years, it is timely to evaluate how the profile of medicine applicants and entrants has changed in response. In addition, it is now recognised that several of these demographic factors intersect in the determination of disadvantage (UCAS, 2018; Social MObility Commission, 2023), yet we do not have a picture of this for medicine applicants and entrants. Accordingly, this project seeks to do three things. First, it will replicate Garrud's 2014 analysis looking at how widely applications have come from schools/colleges in the ten years since SfE - especially looking at how many schools are now contributing at least some applicants, and how many are responsible for the majority of applicants. Secondly, it seeks to provide a picture of the proportion of the 17-18 yr UK school population who apply for medicine that is plotted by ethnicity (e.g. much lower rates of application from white communities, lower from Afro-Caribbean than Black-African - Garrud, Owen, Wei, 2023), gender, and postcode (that will yield both geographical and postcode-based deprivation index maps). This could also include school/college as well, though likely aggregated across the other demographic factors for individual schools. In order to derive sufficiently detailed population information, a parallel application is being made via the ONS Secure Research Service to obtain the relevant aggregated data from the National Census (2011,2021-2) and National Pupil Database (ONS 2023). Thirdly, it will examine the characteristics of those school/colleges that provide no (or very few, unsuccessful) applicants to medicine.

References

  1. Medical Schools Council (2014) Selecting for Excellence Final Report
  2. Garrud (2014) Help and hindrance in widening participation.
  3. UCAS (2018) Multiple Equality Measure
  4. Medical Schools Council (2019) Selection Alliance Annual Report
  5. Garrud, Owen & Wei (2023) Impact of the covid-19 pandemic on UK medical school applications and intake.
  6. Office for National Statistics (2023) Grading and admissions data England.
  7. Social Mobility Commission (2023) State of the Nation 2023: people and places
  8. Wiseman et al (2017) Understanding the changing gaps in higher education participation in different areas of England.

Doctor Paul Garrud
paul.garrud@nottingham.ac.uk
UKMEDP197
Access to medical schools for students from disadvantaged backgrounds

Approved on 01 December 2023 UKMED Advisory Board

Ongoing Project

Historically, UK medical school applications and admissions have had an underrepresentation of students from disadvantaged backgrounds. For example, Steven et al. (2016) found that in 2009-2012 only 2.9% of medical applicants had parents in National Statistics Socio-Economic Classification (NS-SEC) group 5 (semi-routine and routine occupations) compared to 25.2% of the population. Analysing UKMED data from 2007-2014, Kumwenda et al. (2018) found applicants from NS-SEC 5, from the most deprived postcodes, and from minority ethnic backgrounds were less likely to receive an offer than those from more advantaged backgrounds. To address this, medical schools offer foundation and gateway courses that are reaching their goals of attracting applicants from lower socioeconomic status in comparison to standard entry courses (Curtis & Smith, 2020). In 2019 Medical School Council reported that between 2007-2017 there was an increase in medical school entrants from minority backgrounds and from the most deprived postcode areas. However, it did not explore the relative contributions of academic and school characteristics on admissions and only covered the pre-pandemic period. The current study aims to investigate access to medical schools for students from disadvantaged backgrounds from 2012-2022. Within this, we will explore the socioeconomic, demographic, and educational characteristics of English-domiciled applicants, offer-holders, and entrants to medicine nationally, as well as to different medical schools and course types. We will also examine how the likelihood of gaining an offer and entering different types of medical schools and courses varies by applicant characteristics.

References

  1. Curtis, S., & Smith, D. (2020). A comparison of undergraduate outcomes for students from gateway courses and standard entry medicine courses. BMC medical education, 20, 1-14. The paper compares the academic outcomes of students in gateway and standard entry medicine courses, offering insights into how gateway programs impact diversity and academic potential among medical students, aligning with the goals of your proposed study.
  2. Kumwenda, B., Cleland, J., Greatrix, R., MacKenzie, R. K., & Prescott, G. (2018). Are efforts to attract graduate applicants to UK medical schools effective in increasing the participation of under-represented socioeconomic groups? A national cohort study. BMJ open, 8(2), e018946. This study compared graduate and non-graduate applicants to UK medical schools and found only marginal improvements in socioeconomic diversity, suggesting that altering selection criteria for graduates may be necessary to achieve greater diversity. The discussed study is relevant to our proposed research because it highlights the challenges of diversifying medical student populations
  3. Medical School Council (2019). Selection Alliance 2019 Report. An update on the Medical Schools Council’s work in selection and widening participation. Report: https://www.medschools.ac.uk/media/2608/selection-alliance-2019-report.pdf. The Medical School Council report directly examines the demographic characteristics of medical school admissions from 2007-2017.
  4. Steven, K., Dowell, J., Jackson, C., & Guthrie, B. (2016). Fair access to medicine? Retrospective analysis of UK medical schools application data 2009-2012 using three measures of socioeconomic status. BMC Medical Education, 16(11), 11–11. https://doi.org/10.1186/s12909-016-0536-1. This paper highlights the underrepresentation of low SES students in medical school admissions.

Doctor Kathrine Woolf
k.woolf@ucl.ac.uk
UKMEDP198
Understanding the origins of the Scottish GP workforce, especially in hard to recruit areas.

Approved on 01 December 2023 UKMED Advisory Board

Ongoing Project

Scotland needs more General Practitioners (GPs), especially in rural and deprived communities. Policies to address this issue are based on other countries’ interventions, but we do not know whether these work in Scotland. With more knowledge on what influences doctors’ career choices, we could optimise the selection and training of students to make it more likely they will work where they are most needed. This study aims to fill this knowledge gap and provide a base of data to monitor and explore similar issues in the future. The project will use publicly available data on UK doctors and link this to information about their backgrounds and education. Linked data for those who completed UK medical school from 2012 is available in the UK Medical Education Database (UKMED). Doctors who graduated earlier and from outside the UK will be analysed separately. Data will be held and analysed in the UKMED. We ask: 1. What are the differences between doctors who become GPs in Scotland and those who do not? 2. How might experiences and decisions made during medical school and training influence career choice? 3. Are there specific traits that mean someone is more likely to become a GP in a rural or deprived community? We will obtain University ethical approval before work begins. The UKMED ensures we cannot identify individuals and protects their privacy. The team has expertise researching the GP workforce, working with complex data and the UKMED, and has influential networks to create change with the study results.

References

  1. 1. Maclaren AS, Cleland J, Locock L, Skea Z, Denison A, Hollick R, Murchie P, Wilson P. Understanding recruitment and retention of doctors in rural Scotland: Stakeholder perspectives. Geogr J. 2022;188(2):261-276. doi:10.1111/geoj.12439
  2. 6. Wass V, Gregory S, Petty-Saphon K. By Choice - Not by Chance: Supporting Medical Students towards Future Careers in General Practice. 2016. Medical Schools Council; Higher Education England
  3. 7. Scottish Government. Undergraduate Medical Education in Scotland: Enabling More General Practice Based Teaching: Final Report. 2019.
  4. 10. Gale TCE, Lambe PJ, Roberts MJ. Factors associated with junior doctors’ decisions to apply for general practice training programmes in the UK: secondary analysis of data from the UKMED project. BMC Med. 2017;15(1):220. doi:10.1186/s12916-017-0982-6
  5. 12. Kumwenda B, Cleland JA, Prescott GJ, Walker KA, Johnston PW. Relationship between sociodemographic factors and specialty destination of UK trainee doctors: a national cohort study. BMJ Open. 2019;9(3):e026961. doi:10.1136/bmjopen-2018-026961
  6. 14. Alberti H, Randles HL, Harding A, McKinley RK. Exposure of undergraduates to authentic GP teaching and subsequent entry to GP training: a quantitative study of UK medical schools. Br J Gen Pract. 2017;67(657):e248. doi:10.3399/bjgp17X689881

Doctor Kirsty Alexander
kalexander001@dundee.ac.uk
UKMEDP199
Predictive factors for successful recruitment to, and completion of training in Anaesthesia and Intensive Care Medicine in the UK

Approved on 01 December 2023 UKMED Advisory Board

Ongoing Project

The General Medical Council has set a target to eliminate education and training inequalities in the UK by 2031 (1). Both the Royal College of Anaesthetists (RCoA) and the Faculty of Intensive Care Medicine (FICM) are undergoing changing workforce demographics (2, 3) such as an increase in female Consultants, yet anticipate significant upcoming workforce shortages. Female doctors preferentially apply for run-through and non-surgical specialties and, along with white doctors, outperform others in several speciality recruitment interviews (4). Tridente et al (5) demonstrated that being of Asian ethnic origin and having a non-UK medical degree were independent negative predictors of success at Intensive Care Medicine (ICM) training recruitment. We seek to expand on this preliminary understanding by exploring differential attainment with data from a larger cohort and applying this further to Anaesthetics recruitment and progression in both specialties. Through logistical regression models we will investigate the early career performance and socio-demographic factors that predict applicants’ success in core training. We will further determine if the factors continue to be predictive at entry to higher training and completion of Anaesthetics and ICM training. Divergence of under-performing cohorts will be recognised, which we hope will facilitate targeted interventions at medical school, interview and throughout training by recruitment stakeholders to ensure a diverse Consultant workforce. Achieving this will address some of the education and training inequalities in the UK.

References

  1. General Medical Council (2021) GMC Targets Elimination of Disproportionate Complaints and Training Inequalities, Available at https://www.gmc-uk.org/news/news-archive/gmc-targets-elimination-of-disproportionate-complaints-and-training-inequalities (Accessed 25th September 2023) - The GMC outlines their strategy for removing inequality by 2031 in training and recognise the desperate outcomes by ethnic background.
  2. Royal College of Anaesthetics (2022) The Anaesthetic Workforce: UK State of the Nation Report, Available at https://www.rcoa.ac.uk/sites/default/files/documents/2022-02/State-Nation2022.pdf ( Accessed 16th August 2023) - This report recognises the workforce shortages and evaluates the impact of changing socio-demographics will have on future demand, for instance the increasing number of female consultants who are over three times more likely to work less than full time.
  3. The Faculty of Intensive Care Medicine (2021), Workforce Data, Available at: https://www.ficm.ac.uk/sites/ficm/files/documents/2021-10/workforce_data_bank_2021_-_for_release.pdf (Accessed on 10th April 2023) - The FICM report captures that dual-CCT intensivists provide 50% of whole-time equivalent work in intensive care, with over 80% from an anaesthetics background, with increasing single CCT and female consultants.
  4. Santana, I. et al. (2022) A sequential analysis of the speciality allocation process in the UK. Empirical evidence from the UKMED database. Available at https://www.ukmed.ac.uk/documents/reports/UKMEDP22_report.pdf (Accessed 16th August 2023) - Logistic regression analysis of doctor’s characteristics and their influence on performance, recruitment and career progression throughout training using the UKMED database.
  5. Tridente, A. et al. (2022) “Differential attainment and recruitment to intensive care medicine training in the UK, 2018–2020,” BMC Medical Education, 22(1). Available at: https://doi.org/10.1186/s12909-022-03732-w (Accessed 16th August 2023). - Tridente et al’s analysis of two interview cycles for UK intensive care training shows differential attainment with findings of international medical graduate underperformance and UK Asian ethnicity outperformance.

Doctor Hardeep Kandola
hardeep.kandola@nhs.scot
UKMEDP200
The influence of widening participation on medical school admission and progression, career development and dropout

Approved on 01 December 2023 UKMED Advisory Board

Ongoing Project

Five new medical schools were announced in 2018, targeting areas where recruitment into general practice and psychiatry had traditionally been challenging, e.g. Kent. The location of these was predicated on data suggesting that doctors are more likely to remain in the locations where they have trained. These medical schools were anticipated to increase the number of doctors in the local area, improve recruitment to certain specialities, as well as widening the social profile of medical students (1). Improving the diversity of students entering medicine benefits patients in a better understanding of their needs and serves to reduce health inequalities. (2) One of the new medical schools was Kent and Medway Medical School (KMMS) – based in Kent, the largest conurbation in the UK without a medical school. Kent has the lowest number of GPs per capita in the UK and longstanding issues in attracting doctors into the more rural parts of the county.(3) This proposal seeks to understand how widening participation factors (ethnicity, gender and socioeconomic status) of students applying to and studying at KMMS varies in comparison to the frequency of these factors more generally across the UK medical schools. We will also examine how these factors influence students progress throughout UK medical school, their subsequent career choices, and levels of dropout and burnout. More specifically, we will develop a model of how these variables intersect in predicting negative career outcomes and highlight factors that may benefit from early intervention.

References

  1. Health Education England. (2018). New medical schools to open to train doctors of the future. [online]
  2. Health Education England (2014). Widening Participation It Matters! [online]
  3. CEFEUS (2018). Kent and Medway: Health and Social Care a Brexit Impact Assessment

Doctor Joanne Rodda
joanne.rodda@kmms.ac.uk
UKMEDP203
Demographic Factors Influencing Medical Degree Classifications and Subsequent Impact on Post-graduate Training in the UK

Approved on 18 March 2024 UKMED Advisory Board

Ongoing Project

Medical assessments should strive to be equitable, otherwise medical training itself may promote socioeconomic inequality (1). The ‘classification’ of medical degrees beyond a pass (e.g., honours, distinction, merit, commendation) are an assessment outcome that varies between UK medical schools in terms of their criteria and frequency (2), and can be calculated using the UKMED database for each medical school. There is limited research into the impact of these disparities outside the USA, where honours are associated with future career success – yet fewer students from minority ethnicities and socioeconomically disadvantaged backgrounds receive them (1,3). Bias within medical assessments not only pose significant challenges to learners, but may threaten the progression of students into doctors, and beyond (4). Despite the potential impact of the above biases, the ‘consequential validity’ (i.e., the positive or negative consequences) of medical degree classifications in the UK is unknown (5). We hypothesise that similar inequities exist and that classifications are associated with future career success, at medical school and beyond. In this study, we aim to investigate the demographic factors that influence UK medical degree classification and explore subsequent impact on training and progression. In our previous work, we surveyed medical schools for their degree classification criteria, obtaining responses from 36 medical schools (2). These criteria are listed in the following Supplementary Table:[https://asmepublications.onlinelibrary.wiley.com/action/downloadSupplement?doi=10.1111%2Fmedu.15019&file=medu15019-sup-0002-Table+S1.docxdocx]. We propose to leverage the UKMED cohort, combined with our catalogue of classification criteria, to carry out a comprehensive analysis of the impact of demographic factors on the award of classifications at UK medical schools. In addition, we aim to explore impact of degree classification on subsequent postgraduate training performance. Educators can then use knowledge of consequential validity of UK medical degree classifications to inform policy and practice.

References

Doctor Jonathan Wan
Jonathan.wan@ucl.ac.uk
UKMEDP204
Exploration of factors which predict successful entry onto the GP or Specialist registers for doctors with a non-UK Primary Medical Qualification

Approved on 21 June 2024 UKMED Advisory Board

Ongoing Project

The NHS is facing a crisis with a massive shortage of doctors, especially consultants and GPs. In 2022, there were 10,582 vacancies for hospital medical staff across the UK and only 52% of consultant doctor vacancies filled. [1] The UK needs 6,000 more GPs to meet patient needs.[2] International medical graduates (IMGs) are a large and core part of the UK medical workforce, comprising 36% of licensed doctors in the UK in 2021.[3] In 2022, the GMC reported that 52% of newly-registered doctors were IMGs.[3] In 2021, we completed a mixed-methods study examining the drivers and barriers affecting doctors’ migration to and from the UK.[4,5] Very few IMGs join the specialist or GP registers on arrival - just 4.8% did so in 2019 and longitudinal data from 2009 shows that few go on to become consultants or GPs after entry onto the GMC register (11.6% within 5 years and 27.2% within 10 years). In this study we intend to explore individual and educational factors which predict whether IMGs gain specialist or GP registration within 10 years of entry onto the GMC register and to compare the findings with similar analysis for UK graduates. Our research team has completed multiple studies with the UKMED database using similar methodology to that proposed in this study.[6] This study is part of a broader research proposal submitted to the NIHR HSDR, looking to understand how we can support existing international medical graduates working in the NHS to become consultants and GPs.

References

  1. Royal College of Physicians census finds record number of physician jobs unfilled.2022
  2. RCGP. Brief: GP Shortages in England 2023.
  3. General Medical Council. State of medical education in practice: the workforce report 2022
  4. Brennan N, Langdon N, Bryce M, Gale T. Drivers of international migration of doctors to and from the United Kingdom 2021
  5. Brennan N, Langdon N, Bryce M, Gale T. Exploring recent patterns of migration of doctors to the United Kingdom: A mixed-methods study. BMC Health Services Research. 2023
  6. Gale TCE, Lambe PJ & Roberts MJ. Factors associated with junior doctors' decisions to apply for general practice training programmes in the UK: Secondary analysis of data from the UKMED project. BMC Medicine 2017

Thomas Gale
thomas.gale@plymouth.ac.uk
UKMEDP205
Exploring the Progression and Retention of Disabled Doctors in the NHS

Approved on 21 June 2024 UKMED Advisory Board

Ongoing Project

A diverse workforce enhances patient care by mirroring community needs and facilitating person-centred treatment. [1] Supporting disabled doctors is key to NHS diversity, with 24% of the UK population disabled, a figure expected to increase with an ageing, multimorbid population. [2] However, disabled doctors face unique challenges, compounded by poor support, speculated to hinder career progression. [3] Key to ensuring disabled doctors’ valuable contributions to, and presence within, the NHS, is an evidence-based understanding of the career barriers they may face, and the efficacy of available support. However, current knowledge regarding disabled doctors’ careers are fragmented and regional. [4] In addition, the efficacy of available support is unclear. [5] There is need for national exploration of progression amongst disabled NHS doctors, compared with non-disabled doctors, so that targeted and effective strategies can be developed to support and enhance their career progression. This national exploration will provide a clearer understanding of the barriers and facilitators affecting disabled doctors' careers within the NHS. By comparing the career trajectories of disabled and non-disabled doctors, we can identify specific areas where additional support or policy changes are needed. A robust understanding of the nature of progression, and any barriers or challenges, is necessary prior to the creation of evidence-based solutions. We propose to leverage the UKMED cohort to carry out a comprehensive analysis of the impact of being disabled on doctors’ careers. Factors we are interested in include demographic factors, assessment outcomes (including ARCP outcomes), and CCT. Educators, researchers, and policymakers will be able to use this knowledge regarding barriers and enablers for disabled doctors’ careers to inform the development of support strategies.

References

  1. This report provides essential insights into the current status and experience of disabled staff within NHS trusts, crucial for understanding the existing landscape and identifying possible areas for improvement within the study analysis and discussion.
  2. This study offers insights into the impacts of the COVID-19 pandemic on disabled people in Britain, including figures on rates and increases in disability, which are relevant for contextualising the need for this work and in understanding the specific challenges faced by disabled doctors during this period.
  3. This article explores the experiences of disabled doctors, or residents, in the United States, offering insights that are critical for understanding the barriers and facilitators in medical education and training in a different context. It will be relevant to consider these factors within the context of the UK.
  4. This article draws attention to the importance of inclusion of doctors with physical disabilities in surgical training, directly addressing one of the key areas of this study regarding career advancement opportunities. It makes the important point regarding the fragmentation of existing research and support initiatives.
  5. This scoping review provides comprehensive insights into the barriers faced by medical students with disabilities, offering a foundation to understand and analyse similar challenges that may persist into professional practice for disabled doctors.

Doctor Megan Brown
megan.brown@newcastle.ac.uk
UKMEDP206
The Clinical Academic Training Pipeline: Investigating the Impact of Early Career Research Exposure on Recruitment and Assessing Trainee Well-being

Approved on 21 June 2024 UKMED Advisory Board

Ongoing Project

Clinical academics (CAs), who balance teaching, research, and clinical practice, are a small but crucial part of the medical workforce. Recent data from the Medical School Council (1) highlights a concerning trend: that the overall number of CAs has remained largely static since 2005 despite a massive increase in the total number of consultants, such that the proportion of consultants who are CAs is 4% compared with 7% in 2005. In addition, the proportion of CAs aged over 55 has doubled from 18% in 2005 to 36% in 2023. This indicates that a growing segment of the workforce is nearing retirement age without adequate replacements. Maintaining the pipeline of CAs is now more important than ever, particularly in light of the Long-Term Workforce Plan's objective to double the annual intake of medical students to 15,000 by 2032 (2). Furthermore, the start of the pipeline for CAs maybe eroded by recent changes in Foundation Programme recruitment, such as the implementation of the Preference Informed Allocation system along with the elimination of the Educational Performance Measure (EPM), potentially dissuading medical students from pursuing intercalation. Previous research has linked intercalation to a heightened interest in pursuing future academic roles (3-5), although the precise extent of this remains unclear. This study will look at the start of the pipeline and aims to examine how early career exposure to research influences the inclination to enter the academic training pathway. Additionally, we will evaluate potential differences in progression from foundation and specialty training, as well as differences in burnout among academic doctors compared to their non-academic peers. This will begin to assess whether disparities in well-being and retention exist. Moreover, given the underrepresentation of clinical academics from ethnic minority backgrounds or who are female, as indicated by MSC’s survey (1), we will explore whether further variations existing in recruitment and wellbeing when looking by sociodemographic factors.

References

  1. MSC (2024) - Medical schools voice concerns over the drop in the clinical academic workforce
  2. NHSE (2023) - NHS Long Term Workforce Plan
  3. House of Lords Science and Technology Committee (2023) - Clinical academics in the NHS inquiry
  4. McManus et al. (1999) - Intercalated degrees, learning styles, and career preferences: prospective longitudinal study of UK medical students
  5. Nguyen-Van-Tam et al. (2008) - What happens to medical students who complete an honours year in public health and epidemiology?

Miss Emma Fletcher
emma.fletcher@medschools.ac.uk
UKMEDP207
The influence of widening participation on medical school progression, career development and dropout

Approved on 21 June 2024 UKMED Advisory Board

Ongoing Project

Five new medical schools were announced in 2018, which were anticipated to increase the number of doctors in the local area, improve recruitment to certain specialities, and widen the social profile of medical students (1). Widening participation to medicine and improving the diversity of students entering the profession benefits patients in a better understanding of their needs, and serves to reduce health inequalities. (2) Although we are gaining understanding of widening participation in recruitment to medical school, there is less information available on how those students progress, in terms of examinations, degree completion (or dropout) and postgraduate career speciality. Current research focuses on some of these specific areas, but not the whole journey throughout and beyond medical school. There is limited information on the progression of medical students throughout their degree, and the likelihood of degree completion vs. dropout in relation to widening participation factors. We have reviewed UKMED published projects, which have looked at sociodemographic factors and allocation to Foundation Training programmes (3) and specialty destination after the 2 year training programme and found that trainees who came from families where no parent held a degree had statistically significant lower odds of choosing medical specialties and were more likely to choose General Practice.(4) However, as these were studied during and before 2018, they did not include information from the new medical schools with an explicit widening participation agenda. We hope that our work can build upon this. The GMC begun asking questions about burnout in the National Training Survey from 2018 (5) but currently the outcomes have not been linked back to sociodemographic factors.

References

  1. Health Education England. (2018). New medical schools to open to train doctors of the future. [online]
  2. Health Education England (2014). Widening Participation It Matters! [online] Available
  3. Kumwenda B, Cleland JA, Prescott GJ, et al. Relationship between sociodemographic factors and selection into UK postgraduate medical training programmes: a national cohort studyBMJ Open 2018;8:e021329.
  4. Kumwenda B, Cleland J, Prescott G, et al. Relationship between sociodemographic factors and specialty destination of UK trainee doctors: a national cohort studyBMJ Open 2019;9:e026961.
  5. General Medical Council. (2022). National Training Survey 2022 results. [online]

Professor Sukhwinder Shergill
sukhwinder.shergill@kmms.ac.uk
UKMEDP208
Does performance at school, university, and post-graduate level effect successful application to Clinical Radiology and subsequent performance on the Fellowship of the Royal College of Radiology (FRCR) examinations?

Approved on 21 June 2024 UKMED Advisory Board

Ongoing Project

There is an increase in Radiology workload and subsequent demand for Radiologists in the UK [1]. To become a Consultant Radiologist in the UK, doctors must first apply to Radiology specialty training and subsequently pass the Fellowship of the Royal College of Radiology (FRCR) examinations. Competition ratios for Radiology specialty training is on an exponential rise [2], and the Radiology Royal College exams are anecdotally considered to be some of the most difficult to pass. To date, little evidence exists exploring the factors that drive/contribute to successful application to Radiology training and passing the various FRCR examinations. We aim to fill this gap by exploring the possible factors that influence successful application to Radiology training and passing of the various FRCR examinations, such as background demographics, performance at school, medical school factors, performance in Foundation Programme selection, and performance on postgraduate examinations prior to sitting the FRCR exams (e.g. MRCP and MRCS).

References

  1. RCR. Turnaround times – what are we seeing?
  2. Escalating competition in NHS: Implications for Healthcare Quality and Workforce Sustainability.

Doctor George Choa
george.choa@nhs.net
UKMEDP209
Factors influencing short- and long-term retention of UK medical graduates in the NHS.

Approved on 21 June 2024 UKMED Advisory Board

Ongoing Project

The UK faces a significant shortage of doctors (1). While both main political parties seek to address this through a large increase in medical student numbers (2), we think it also important to address retention of doctors in the profession, particularly in light of the recent large AIMS study (led by the one of the team (TF) of this proposed study), which reported that over a third of current medical students plan to abandon the NHS within two years of graduation (3). We think it essential to understand major factors which influence retention, so we can then address these. The planned study would complement the AIMS study, with two major advantages: it will demonstrate actual, not intended, outcomes; and there will be no selection bias given full data for doctors still registering in the UK are available through UKMED. This needs to be understood alongside the major limitation: it demonstrates outcomes of past graduates, so may not apply to future graduates. We propose to test the effects of two exposures of interest: 1. Ethnicity. Multiple studies have demonstrated awarding gaps in undergraduate and postgraduate medical exams between White student/doctors and minoritised Students/Doctors of Colour (4, 5). Several mechanisms have been proposed to explain this, including differences in learning environment and experiences of racism (6). It is important to test whether this also feeds through to lower retention in the profession: however, empirical evidence for this is lacking (7). 2. Graduate vs Standard courses. Entrants to graduate courses already have a degree and are, on average, older. Taking a second degree at an older age may demonstrate greater commitment to medicine, but having another degree may also lead to greater career opportunities outside medicine. A major difference between retention rates from different course types should be taken into account when planning medical school expansion. We propose to test outcomes at three timepoints after graduation: three, six and twelve years.

References

  1. 1) NHS England. NHS Vacancy Statistics England, April 2015–March 2023, Experimental Statistics [Internet]. 2023 May [cited 2024 May 01]. Available from: https://digital.nhs.uk/data-and-information/publications/statistical/nhs-vacancies-survey/april-2015---march-2023-experimental-statistics
  2. 2) NHS England. NHS Long Term Workforce Plan [Internet]. 2023 Jun [cited 2024 May 01]. Available from: www.england.nhs.uk/publication/nhs-long-term-workforce-plan/
  3. 3) Ferreira T, Collins AM, Feng O The AIMS Collaborative, et al. Career intentions of medical students in the UK: a national, cross-sectional study (AIMS study). BMJ Open 2023;13:e075598. doi: 10.1136/bmjopen-2023-075598
  4. 4) Brown C, Goss C, Sam AH. Is the awarding gap at UK medical schools influenced by ethnicity and medical school attended? A retrospective cohort study. BMJ Open 2023;13:e075945. doi: 10.1136/bmjopen-2023-075945
  5. 5) Woolf K , Potts HWW , McManus IC. Ethnicity and academic performance in UK trained doctors and medical students: systematic review and meta-analysis. BMJ 2011;342:d901
  6. 6) Woolf K. Differential attainment in medical education and training BMJ 2020; 368 :m339 doi:10.1136/bmj.m339
  7. 7) Woolf K, Papineni P, Lagrata S, Pareek M. Retention of ethnic minority staff is critical to resolving the NHS workforce crisis BMJ 2023; 380 :p541 doi:10.1136/bmj.p541

Professor Paul Wilkinson
pow12@medschl.cam.ac.uk
UKMEDP210
Does socio-economic status influence medical school applications and success in being offered a place: a retrospective cohort study in England from the 2022 admission cycle.

Approved on 21 June 2024 UKMED Advisory Board

Ongoing Project

This research aims to investigate which measures of socio-economic status are associated with each of the three stages of a medical school application. These stages include; application to medical school, being offered a place and taking up a place. This research will be focusing upon non-graduate students from England aged 20 and under, applying to standard entry medicine programmes at UK medical schools from the 2022 cycle. The measures of socio-economic status to be tested in this project – NS-SEC, POLAR 4, IMD/NIMDM/SIMD/welsh IMD, school type, parental occupation and employment status, IDACI, Adult HE quintile, TUNDRA, average point per A-level student for school attended. The data will be analysed used multiple logistic regression analysis with a p value of 5%. All data analysis will take place within SPSS. Using the association between these measures of socio economic status and the probability of applying, being offered a place and then accepting that place. We will infer which measures of socio-economic status are the best predictors of medial school admission at each of these stages. This is important because medical schools are frequently using measures of socio-economic status to assist in selection of candidates for interview. It is therefore imperative that we are using the correct measures to ascertain which students are from less affluent backgrounds to assist in widening participation to medical school.

References

  1. This study reviewed multiple factors influencing medical school admission including socio-economic status. They found that the application process was less successful for those from lower socio-economic backgrounds. This study was based in Australia and only looked at IRDS as a measure of socio-economic status. We plan to conduct our research on further measures of socio-economic status and also do this within England, the methodology which we propose to use within our research will be similar to this study.
  2. This paper reviewed 3 measures of socio-economic status and medical school admission. It concluded that IMD may be misleading. Our research aims to build upon this work and evaluate more measures of socio-economic status. Furthermore, this study was conducted on the 2009 to 2012 cohorts, we aim to evaluate more recent cohorts to see if any progress has taken place in encouraging widening participation in those from lower socio-economic backgrounds and to also evaluate newer measures.
  3. This paper reviewed multiple measures of socio-economic factors to determine a widening participation measure score.
  4. This research looked at social class as inequalities in medical school admission. This research was conducted on UCAS data from 1997-2000. We aim to look at further measures than this paper, but also see if there has been any improvement in widening participation over the last 2 decades.

Doctor Wajeeha Aziz
w.aziz@bsms.ac.uk
UKMEDP211
Can previous academic achievement be used to predict risk of not completing paediatric speciality training?

Approved on 21 June 2024 UKMED Advisory Board

Ongoing Project

The NHS paediatric workforce is facing a crisis. The recruitment of junior doctors to paediatrics is falling resulting in rising rates of unfilled training posts and a consultant shortage1,2. This is combined with poor rates of employee retention, with 9.1% of NHS employees leaving the workforce in 20223. The Royal College of Paediatrics (RCPCH) has stated this poses “significant risks” to children’s health and project this will only worsen as demands on paediatric services rise1,2. In light of this the current literature, NHS workforce plan and RCPCH have recognised the need to bolster the paediatric workforce via improved recruitment to training and enhanced employee retention3-5. Despite this, there is a paucity of research into understanding the demographic backgrounds and academic achievements of junior doctors entering and leaving paediatric training3,4. This creates difficulty identifying the success of current recruitment strategies and those at risk of leaving training. Without this fundamental knowledge, achieving the workforce expansion seems unlikely. The UKMED database therefore provides an opportunity to gain an insight into those entering, completing and prematurely leaving paediatric training. The authors aim to go one step further and explore if previous academic achievement can be used to predict a doctor’s risk of having issues completing, or prematurely leaving, training. If this relationship exists, it will allow for the identification of doctors who may benefit from increased mentoring, thereby providing a practical means of employee retention. Overall the findings of study would provide the foundations on which the paediatric workforce expansion could be achieved.

References

  1. RCPCH. RCPCH Medical Workforce Census 2015. 2017
  2. RCPCH. RCPCH Workforce Briefing 2018
  3. NHS England (2023). The NHS Long Term Workforce Plan
  4. 4. Redman MG, Carzedda D, Jay N, Clark SJ, Rogers M. Searching for the true attrition rate of UK paediatric trainees. Archives of disease in childhood. 2021 Sep 1; 106(9). pp903-5
  5. RCPCH. RCPCH Vision 2040

Doctor Jonathan Darling
j.c.darling@leeds.ac.uk
UKMEDP212
Exploring attrition from speciality training programmes in the NHS

Approved on 21 June 2024 UKMED Advisory Board

Ongoing Project

Each year 5000 junior doctors in the UK start speciality training (ST) with a view to becoming a hospital consultant. But despite a competitive selection process, not all doctors finish training. This is a problem, because it is a costly loss of potential in the NHS, (1) and risks doctors’ wellbeing and patient care. (2) In this study we use ‘attrition’ to mean doctors leaving ST before completion. Some will leave due to performance issues (e.g., exam failure); others decide to leave during training, and some decide not to apply for consultant posts. Data from the General Medical Council (3) indicate an average of 800 trainees leave each year, with some regions and specialties more affected than others. (4,5) We are interested in attrition, rather than in other more diffuse issues (such as doctors leaving the NHS to migrate or retire early, as a decision to leave is a singular event, and so more open to causal analysis and mitigation through targeted interventions. This study aims to understand the proportion of trainees do not complete ST, and how the risk of trainees not completing ST varies over time, with system and individual factors (including training programme, location, and demographic factors). Educators, researchers, and policymakers will be able to use this knowledge regarding attrition to inform the development of support strategies to improve retention throughout speciality training.

References

  1. This report from the BMA highlights the potentially monetary cost of attrition, with estimates that the cost to replace a 1st year speciality trainee in psychiatry, (including administrative costs, cover costs, productivity costs, and training costs) has a minimum figure of £230,000.
  2. The workplace experience report (2023) highlights high levels of dissatisfaction amongst the UK workforce, including indicating that increasing numbers of doctors are considering leaving.
  3. The GMC Data Explore provides some detail about attrition from speciality training in the UK. Our proposed analysis complements this through a longtitudinal analysis, which enables us to link demographic and geographical factors.
  4. This RCP report shows an average attrition rate of 5% in medical specialities, with direct implications on rota instability and perceived threats to patient safety.
  5. This paper, which utilised FOI requests to deaneries, shows how the attrition rate of paediatric trainees is difficult to calculate.

Doctor Hannah Gillespie
hannah.gillespie@newcastle.ac.uk
UKMEDP213
GPs workforce labour transition

Approved on 29 November 2024 UKMED Advisory Board

This research aims to investigate the significant shortage of GPs in the UK and the disparities in GP numbers per capita across regions by linking medical student training data with the workforce trajectories of GP doctors. The study will address three key questions: the factors influencing practice choice among new entrants, such as gender and place of qualification; the impact of these choices on inequalities in primary care provision, assessed through Quality and Outcomes Framework (QOF) scores and patient satisfaction; and the effectiveness of policies like Targeted Enhanced Recruitment Schemes (TERS) and scholarships in reducing GP per capita inequality. By identifying the key determinants of practice location and their implications for care quality, this research will provide valuable insights for policymakers aimed at addressing regional disparities in healthcare access. The findings will help inform targeted interventions to attract new GPs to underserved areas, ensuring equitable distribution of healthcare resources and improving overall patient outcomes across the UK.

References

  1. Study 1: Marchand, C., & Peckham, S. (2020). "Addressing the Crisis of GP Recruitment and Retention: A Systematic Review": This systematic review examines the factors contributing to GP recruitment and retention challenges, with a focus on new entrants. The authors analyze various studies to identify common barriers faced by new GPs, such as workload, job satisfaction, and access to support.
  2. Study 2 : Nussbaum et al, (2021): Inequalities in the distribution of the general practice workforce in England: This is a practice-level longitudinal analysis longitudinal analysis. - Significant workforce inequalities exist and are even increasing for several key general practice roles, with workforce shortages disproportionately affecting more deprived areas - Policy solutions are urgently needed to ensure an equitably distributed workforce and reduce health inequities. The paper by Nussbaum et al. (2021) is relevant to the proposed study as it highlights the significant workforce inequalities that exist in the general practice workforce in the UK and the need for policy solutions to ensure an equitably distributed workforce and reduce health inequities.
  3. Study 3: Costa, J., Nunes, L. M., & Sánchez, F. (2018). "How to Attract Physicians to Underserved Areas? Policy Recommendations from a Structural Model": This paper develops a structural model to analyze policies aimed at attracting physicians to underserved areas. By estimating the preferences of doctors for different locations, the model evaluates the effectiveness of financial incentives, such as bonuses or targeted scholarships. The findings suggest that targeted incentives can have a significant impact on reducing regional disparities in physician distribution, especially when coupled with broader measures to improve working conditions.

Doctor Catia Nicodemo
catia.nicodemo@economics.ox.ac.uk
UKMEDP214
DiffErential attainment and Factors AssoCiated with Training applications and Outcomes (DE FACTO): A post-COVID update

Approved on 29 November 2024 UKMED Advisory Board

Core Surgical Training (CST) and Internal Medical Training (IMT) are the first hurdles in higher speciality training (HST) in UK. There are worsening competition ratios for all of both CST/IMT and HST, having nearly doubled from 2019 to 2024 [1,2]. Current literature has demonstrated there is obvious disparity between sexes when it comes to surgical training. Given that 59% of medical students and 54% of foundation trainees are women, it is surprising to see that upon entering the next stage of training, only 41% and 12% of those that secure CST and consultant posts are women respectively [3,4]. Moreover, a previous 2011 study showed that HST surgical posts are more likely to be offered to recently qualified, UK-trained, white, male applicants [5]. Our more up to date studies across different surgical specialties reinforced these demographic, socioeconomic and educational factors that influence successfully obtaining HST posts [6,7]. However, this research was conducted pre-COVID and all these papers highlight the need to provide a post-COVID update to the existing literature. Further, given the paucity of literature understanding the role of factors in medical specialities, our investigation will include group 1 and 2 medical specialities as well. Our aim for wanting to investigate these factors is two-fold: to describe current population characteristics of applicants who apply to and successfully obtain HST posts, and identify areas that need further attention to promote equality, diversity and inclusion in medicine and surgery. We would use similar methodology to our previous EDFACTO papers, and two studies (8,9).

References

  1. NHS England. Competition ratios for 2024, NHS choices.
  2. NHS England. Competition ratios for 2019, NHS choices.
  3. Number of women entering medical school rises after decade of decline
  4. A fifth of surgeons in England are female.
  5. Core surgical training and progression into specialty surgical training: how do we get the balance right?
  6. DiffErential attainment and Factors AssoCiated with Training applications and Outcomes (DE FACTO) study: Trauma & Orthopaedic surgery in the UK.
  7. Differential attainment and factors associated with training applications and outcomes: Urology in the United Kingdom.
  8. Effect of sex on specialty training application outcomes: a longitudinal administrative data study of UK medical graduates.
  9. Predictors of ophthalmology career success (POCS) study.

Doctor Prakrit Kumar
prakritraj.kumar@nhs.net
UKMEDP216
The predictive validity of selection for entry into postgraduate training in Core Anaesthetics Training: A retrospective cohort study

Approved on 29 November 2024 UKMED Advisory Board

Selection into Anaesthetics training has consisted of an interview and the Multi-Specialty Recruitment Assessment (MSRA) for a number of years. The MSRA is a reliable and standardised computer-based assessment used by several specialties, including Anaesthetics. Previous research has demonstrated that performance on the MSRA validly predicts future performance in relation to GP training (Patterson et al., 2013; Patterson et al., 2015), over and above interviews. While there is no published research into the predictive validity of the MSRA in Anaesthetics training, as yet unpublished findings from analyses of data drawn from the UKMED have supported the validity of the MSRA in Anaesthetics (UKMEDP154). Previously, the value of face-to-face selection processes were raised in a review of GP selection (Davison et al., 2016). Therefore, the aim of this study is to further examine the incremental value of each element of the selection process, for recruitment into Anaesthetics training. The data will be used to examine, in more depth and with a larger sample, the relationships between performance in selection and subsequent in-training outcomes (including FRCA examinations and ARCP). This will provide a better understanding of the potential added value of different selection methods in this context. We will explore group differences on the MSRA and interview, to understand how the selection methods perform from an EDI perspective. This would inform future discussions around use of the interviews and MSRA in Anaesthetics selection. For example, whether only certain applicants should be prioritised for interviews, using a cut and/or bypass score.

References

  1. Evaluation of GP Specialty Selection in the UK, including the MSRA
  2. Longitudinal predictive validity of the GP selection process in the UK
  3. Predictive validity of the anaesthetics core training recruitment process in the UK
  4. Predictive validity of the selection methods used for recruitment into UK psychiatry core training

Professor Fiona Patterson
f.patterson@workpsychologygroup.com