9 NOVEMBER | Short presentations Professionalism, Postgraduate and Diversity Theme

* Erika Marek, Dora Schmel, Timea Nemeth, Nora Faubl, Zoltan Katz, Istvan Szilard
* Corresponding author: Erika Marek, Department of Operational Medicine, Medical School, University of Pécs, Hungary
erika.marek@aok.pte.hu

Background
Since 2013 the rapidly growing international migration has posed new challenges to EU healthcare systems. Due to the crisis situation in 2015 a „High-level meeting on Refugee and Migrant Health” was organised by WHO and its Outcome Document emphasizes the importance of providing migrant-sensitive health care and training health professionals taking into account the migrants’ cultural, religious, and linguistic diversity. Along these recommendations this study aims to assess medical students’ actual level of intercultural competence (ICC), and to identify potential deficiencies and areas to improve.

Summary of Work
A questionnaire survey was conducted at University of Pécs, in 2017 spring semester involving 244 4th-year medical students of the Hungarian, English and German training programs. Self-administered questionnaire was used with 94, mainly close-ended items. The tool was a modified and translated version of ‘Clinical Cultural Competence Questionnaire’ developed by R.C. Like. The items were grouped into 6 sections: demography, knowledge, skills, attitudes and comfort-level in intercultural situations, and trainings. Data were analysed using SPSS 21.0.

Summary of Results
No significant difference was found between sex, and age regarding the 4 main examined domains: 'Knowledge', 'Skills', 'Attitudes' and 'Comfort-level in Intercultural Situations'. Compared by study programs, the students of the German Program demonstrated significantly higher scores for 'Knowledge' domain, while students of the Hungarian Program had higher scores for the 'Skills' domain. The longer time interval one has spent or lived abroad showed higher ‘Comfort-level’ scores, but –unexpectedly- showed no relation with better intercultural knowledge, skills and attitudes. Significantly higher scores were clearly demonstrated in terms of ’better language competencies’ for all of the 4 examined domains, while ’previously completed ICC education’ related to a higher ’Knowledge’ scores but had no effect on participants’ ’Skills’, ’Attitudes’ and ’Comfort-level’.

Discussion and Conclusions
Based on our results, we concluded that future training programs at medical schools, in addition to improving students’ awareness, shall focus more on developing skills and enhancing attitudes using various interactive teaching methods. Our study highlighted some specific areas to consider during future curriculum developments, such as ’providing culturally-sensitive end-life care’ as our study participants demonstrated the lowest ‘Skill’-scores regarding this issue. Another important area to focus on is to improving students’ coping strategies when dealing with derogatory remarks on ethnicity during care (both on their own and others’ ethnicity, including the patients). It is encouraging, that study participants considered it important to offer ICC training for healthcare workers, and a great majority (80%<) of them expressed interest and willingness to participate on future trainings aiming to improve their intercultural competencies.

Take-Home Message
Assessing the baseline level and deficiencies in medical students’ intercultural competence may serve as strong evidence for the development of future specific and targeted ICC education. This research was supported by János Bolyai Research Scholarship of the Hungarian Academy of Sciences. The support period of this stipend is 2017/09/01-2020/08/31. „Supported by ÚNKP-19-4 New National Excellence Program of the Ministry of Human Capacities” and University of Pecs, Medical School, PTE ÁOK-KA [No.2017-16].

Summative assessment of professionalism in an authentic, integrated clinical skills assessment

* Elizabeth Metcalf, Sinead O'Mahony, Saadia Tayyaba, Rhian Goodfellow
* Corresponding author: Elizabeth Metcalf, Cardiff University, United Kingdom
metcalfep@cf.ac.uk

Background
The ISCE (Integrated Structured Clinical Examination) was designed to provide evidence for judgements of student performance, and to foster learning, based upon rich feedback given to students following the exam. We value the fully integrated nature of the assessment of professionalism within the ISCE. Five equally weighted ‘domains of competence’ give credit to students who demonstrate global competence & fluency of the task:
• Clinical examination- PROCESS
• Clinical examination- CONTENT
• Diagnostics and clinical reasoning
• Clinical care and patient safety

Professionalism
Prior to 2017/18, all summative marking was undertaken by trained Examiners, with the addition of formative feedback from simulated patients regarding demonstrated communication skills. After the release of results, personalised and constructive student feedback is given which links individual’s results to overall assessment criteria and intended learning outcomes.

What we did
In 2017/8, in order to improve the quality of feedback relating to student’s demonstrated professionalism, we introduced:
1. Examiner feedback: in addition to existing feedback (what was done well and what could be improved), examiners were asked to provide free text feedback regarding students’ demonstrated professionalism.
2. Simulated patient (SP) feedback: a formative domain mark was added, assessing student’s demonstrated professionalism, from the simulated patient’s perspective.

In 2018/19, we additionally included feedback from the perspective of real patients involved with the ISCE, asking for their formative rating of the students’ demonstrated professionalism.

What we found
Results from 2017/18 showed that including SP rating at station level did not have any significant negative impact on students’ performance. The overall difference in SP vs examiner marking was not statistically significant and Examiners’ rating and formative SP rating of professionalism showed a significant and positive correlation. Reliability of the station marking remained consistent with both examiner and SP marking for most of the stations. No significant differences were found in SP vs examiner rating according to students’ gender, ethnicity and language groups. As a result, the pilot was extended in summer 2019 to include real patients. Detailed qualitative analysis is underway to identify key elements of examiner perception of student professionalism within the ISCE. Real patient data from the Year 4 ISCE was collected in May 2019 and further data will be collected in the Year 2 ISCE in June 2019. Psychometrical analysis will then be undertaken to explore the impact of real patient assessment on overall results and reliability of the ISCE.

Conclusion
Focusing upon professionalism as an integrated component of clinical competence within a summative assessment of clinical skills resulted in improved quality of student feedback, embracing the patient voice and improving the authenticity of the assessment. Furthermore, introduction of summative assessment by real patients is now being investigated with a view to inclusion of a summative assessment in the future.

How ready are we for change? Reviewing assessment practices in postgraduate medical education at a medical school in southern India

* Kadambari Dharanipragada, Janet Grant, Danette McKinley
* Corresponding author: Kadambari Dharanipragada, Jawaharlal Institute of Postgraduate Medical Education and Research, India
kadambari1909@gmail.comm

Background
In addition to training opportunities, a well-planned assessment system is needed to ensure achievement of competence as well as progress towards it. This study was done at the Jawaharlal Institute of Postgraduate Medical Education and Research (JIPMER), a medical school in southern India as an initial step towards improving current assessment of postgraduate trainees. The perspectives of the two main stakeholders, teachers and trainees, were sought to develop consensus on the need for change.

Summary of work
In that context, the following questions were asked:
1. What are the views of teachers and trainees on the current assessment practices in postgraduate programmes at JIPMER?
2. What changes do the teachers visualise in assessment that would ensure assessment of competence and fitness to practice?
3. What changes in the assessment practices, according to the trainees, would help them maximise their learning and performance?
4. What are the commonalities and differences in the views expressed by teachers and trainees?

Methodology
Two hundred and eighty-six second- and third-year postgraduate trainees and trainees who had just completed training were invited to participate in an anonymous online or paper-based questionnaire survey containing closed and open ended questions. Fifty-five teachers from various specialties participated in 6 focus group discussions. Written informed consent was obtained from all. The questionnaire and triggers for the focus group discussions were piloted for clarity and consistency. The data was independently coded by the investigators and consensus reached by discussion. Responses to common questions for the two groups were compared. To identify differences in the proportion of teachers and trainees responding, z-scores were calculated for each factor.

Summary of Results
There was a response rate of 50% from trainees. Both teachers and trainees listed a range of assessments currently employed. 43% of trainees said however that they were rarely or never assessed on their day to day work. Both agreed that more weightage needs to be allotted to logbooks and that periodic assessments were needed that must count towards summative decisions. Teachers felt the need for assessments to test critical thinking, capacity to handle complex situations and present arguments that would reflect competence and fitness to practice Trainees wanted more emphasis on assessment of attitudes, surgical skills (among the surgical trainees) and day to day work, clear yardsticks for performance across the training period and greater mentorship by faculty who could provide remedial, personalized feedback.

Discussion & Conclusions
The results indicate that the current assessment of postgraduates at JIPMER do not adequately address the required competencies. Change is needed in the content and tools used for in-training assessment, with the employment of a range of tools for different purposes. The next step in the change process would be to disseminate the results of this study among other stakeholders including other teachers and the administration to bring about an institutional change in the assessment system.

Take home message
The assessments currently being conducted in postgraduate medical education at JIPMER require improvement in terms of content and structure.

Does the GMC Survey really help improve standards? Analysis of changes over time.

* Mathavi Uthayanan, Jonathan Round, Anand Mehta, Geeta Menon,
* Corresponding author: Mathavi Uthayanan, Health Education England and St Georges University Hospital NHS Trust, United Kingdom
mathavi.uthayanan@nhs.net

Background
The General Medical Council National Trainee Survey (GMC NTS) is carried out yearly to monitor training experience across the UK and has a high uptake. Its relevance is not uniformly recognised, with issues around sample size, granularity and wording of the questions. Nevertheless, it is used by Health Education England as one of the cross-programme measures of quality and prompts interventions to departments with poor scores. A discussion at a regional meeting on the reaction to ‘red signals’ led to this more systematic exploration on the value of the survey at improving performance.

Summary of work
The publicly available database of the GMC NTS converts the trainee survey results into a colour scheme, ranging from Green, Light Green, White, Pink, Red, Yellow and Grey. Yellow and Grey are provided for incomplete or not enough trainees, for that indicator. Red indicates, the average score and its confidence interval, fall within the bottom quartile, whilst Green on the other end indicates, the average score and its confidence interval, fall within the top quartile. In our example case, the number of red scores were summated per department and the department with the highest number of reds in 2016 was selected for improvement. Several innovative interventions such as; reorganising the senior staff to allow better access to educational opportunities, better supervision and introduction of a teaching programme, were put in place.

Summary of results
The 2017 GMC NTS results demonstrated the positive correlation between the effect of identifying a struggling department and the result of these timely interventions. Taking this a step further, applying the student’s T test demonstrates a statistically significant difference (probability of 0.038) between the average scores within this same department from 2016 to 2017.

Discussion and Conclusion
The GMC NTS tool is currently in widespread use within the management of medical education, and there is a lack of information available with regards to its use and impact at individual trust levels. We have looked at a particular department at one site, which has scored poorly in 2016, according to the GMC NTS. Put in place multiple innovative interventions, and then used the GMC NTS results from 2017, to assess the effects of these interventions. Furthermore, the results from 2018 are encouraging, with zero red scores in this particular department and indicates sustainability of these interventions. Take home message: GMC survey is a powerful tool used by Health Education England, along with other tools, to assess and improve the standard of medical education within trusts across the UK.

References
1 – Bring on the Reds – an Alternative Guide to the GMC Trainee Survey (date viewed 17/06/2019)
2 –GMC National Training Surveys (date viewed 17/06/2019)

Representation of LGBTQ+ people in medical assessment and teaching scenarios

Emma Alexander, Ronan Lyne, * Yusuf Mottaghi-Taromsari, Isabel Lever
* Corresponding author: Yusuf Mottaghi-Taromsari, Homerton University Hospital NHS Trust, United Kingdom

Background
Examinations and teaching at medical school in the UK are meant to be targeted towards common clinical scenarios faced by junior doctors in the UK National Health Service (NHS). This should include preparing students to provide equal treatment to groups with protected characteristics, including lesbian, gay, bisexual, transgender, queer + (LGBTQ+) people. However, existing research has implied such groups face discrimination when receiving healthcare. This research project aims to: (1) Explore whether LGBTQ+ people are adequately represented in assessment and teaching scenarios; (2) Explore whether medical students feel comfortable discussing sexuality in clinical scenarios; (3) Explore whether prejudiced attitudes towards LGBTQ+ people are present in medical settings.

Summary of work
Participants were current medical students at all stages of training. Participants completed a survey on Google Forms in mid-2018, distributed on public forums (year group pages, MSA Newsletter). The questions were developed both with a Likert scale (1-5, 1 = Strongly Disagree and 5 = Strongly Agree) and with free text space where applicable. The data analysis used descriptive statistics on the questionnaire responses. This study received Minimal Risk clearance (Reference MRS-17/18-8128 and MRS-17/18-8129).

Summary of results
A total of 107 medical students participated. The majority (81%) attended one London-based medical school. The most common stage of training was in the final clinical year (38%). Overall 72% of students had been taught using educational scenarios including LGBTQ+ people, but fewer remembered LGBTQ+ representation in OSCEs (21%) and written exams (29%). When asked whether LGBTQ+ people were adequately represented in assessments and teaching scenarios, the median response was 2, Disagree. When asked whether LGBTQ+ people most commonly appear in scenarios when the diagnosis is related to sexuality or sexually transmitted diseases, the median response was 4, Agree. Medical students were generally comfortable with discussing sexual orientation (Median response 4, Agree). Participants generally felt medical school was an accepting environment for LGBTQ+ people (Median response 4, Agree) but felt less confident about the NHS working environment (3, Unsure).

Discussion & Conclusion
Our study shows there is limited overt representation of LGBTQ+ people in medical school teaching and assessment scenarios. Current representation predominantly focuses on sex/STD-related scenarios.

Take home message
Further work should be done to update curricula and examinations to ensure adequate representation of LGBTQ+ people at all stages.

Creative collaboration: producing authentic OSCE stations to assess professional behaviours

William Curnow, * Suzanne Chamberlain
* Corresponding author: Suzanne Chamberlain, General Medical Council, UK
suzanne.chamberlain@gmc-uk.org

Background
The introduction of the Medical Licensing Assessment (MLA) in the UK will create a requirement for medical schools to demonstrate summative assessment of professional behaviours in their final clinical skills assessment. As part of the planning for the introduction of the MLA, in 2018 the General Medical Council (GMC) commissioned independent research to explore best practice in summative assessment of professional behaviours in clinical skills assessment. The research highlighted that professional behaviours are typically assessed longitudinally, formatively, programmatically, and in a variety of contexts, including workplaces.

Summary of work
In response to the research findings, the GMC developed a series of workshops with a small group of medical educators from UK medical schools. The purpose of the workshops was to explore and experiment with station ideas to see if they could be worked up to feasible OSCE stations with a primary focus on assessing professional behaviours. These included station scenarios about challenging interactions with patients’ carers/relatives, dealing with a clinical error, and decision-making around end of life care. It was envisaged that the participants would co-produce a small pool of OSCE stations for sharing with medical schools, and which schools could use or adapt for their own final clinical skills assessment. Agency role players also attended to test the station ideas and contribute to their development.

Discussion
This paper presents the aims of the workshops and outlines how they were convened and facilitated. Some of the outputs of the workshops and the feedback received from participants will also be discussed. The process has produced a number of authentic clinical scenarios that have been pre-tested and are ready for piloting. Feedback from participants suggests that the collaboration between peers and working with role players have been central to transforming initial scenario ideas about complex interactions to a small number of stations that may prove useful to medical schools. It is anticipated that the workshops will continue for the purposes of generating and sharing the outputs, as well as developing expertise in writing professional behaviour stations for summative assessment.

Take-home message
Working with peers and simulated patients can help overcome some of the challenges of producing OSCE stations to assess professional behaviours.