Saturday 9 November - Academic Short Presentations (EBMA)
10:50-12:30, Room: 1.20, 2nd floor
Chair: Professor Brian Lunn, Newcastle University
Is clinician gaze and body language associated with their ability to identify safeguarding cues?
* Abigail Powell, Nicholas Schindler, Caroline Fertleman, Xueni Pan
* Corresponding author: Abigail Powell, UCL, United Kingdom
One in five children experience neglect at home, school or in the community. They are likely to come into contact with primary care providers, providing a potential safeguarding opportunity for recognition and management. Virtual reality has traditionally been used in medicine to teach surgical skills and is now being harnessed to create high fidelity scenarios for empathy and communication training.
Summary of work
This study focuses on virtual reality consultations between a real GP and an avatar father and son. The behaviour of the father should elicit concern. We investigated whether the GP's body language and the direction of their gaze was associated with an ability to identify safeguarding cues. GP's body language was measured in terms of their head nod, body lean and eye contact. Gaze was assessed as the proportion of the consultation the clinician was looking at the child. Structured interviews were carried out with participants one year on from the original participatory study to explore their experience, and its usefulness as a training tool.
Summary of results
Among 61 participants there was a strong, statistically significant association between empathetic body language and an ability to identify safeguarding cues. There was also a weak association between child-directed gaze and an ability to identify safeguarding cues, but this did not reach statistical significance. Of the twelve GPs interviewed, some felt that virtual reality was a useful training tool due to its accessibility, whilst others felt it removed the human aspect from medicine. There was a consensus that in its current iteration the interface was not reactive enough, and that the poorly rendered graphics were a barrier to interaction with the avatar, hindering immersion.
The findings of this study are in-keeping with previous work suggesting clinicians with more empathetic body language establish a more emotional connection with their patients. However, some interviewees noted that their own non-verbal communication was impeded by the low graphical fidelity and difficulty interacting with the avatars. This raises the question as to whether empathetic body language was more of an indicator of immersion in the scenario, rather than emotional interaction with the avatar. The association between direction of gaze and ability to identify safeguarding cues did not reach statistical significance. The gaze analysis in this study was not sophisticated; only the time spent looking at the child was measured, rather than analysis of dwell time and fixation count. It was in these metrics the greatest differences were found in previous studies.
This study has shown an association between empathetic body language and an ability to identify safeguarding cues within a virtual reality consultation. Whilst this study cannot be used to determine why this association exists, previous research suggests it may be due to higher emotional and cognitive processing, or greater immersion in the scenario. Further research is required to explore this association and determine if education and assessment of clinician body language and gaze can influence the ability to identify safeguarding concerns in children and young people.
Minho Communication Assessment Scale: development and validation of a new tool
* Nuno Goncalves, Monica Goncalves, Pedro Morgado, Nuno Sousa, Patrício Costa, João Cerqueira
* Corresponding author: Nuno Gonçalves, School of Medicine, University of Minho, Portugal
Verbal and non-verbal communication skills are considered core competencies in medical practice and are critical for healthcare outcomes. These skills should be acquired and monitored at medical schools, but their assessment poses a challenge. The aim of this work is to develop and assess the psychometric properties, validity and reliability of a communication assessment scale (CAS).
Methods and Results
In this study, we developed a CAS with 23 items, composed by 5 dimensions (Structure, Way of Questioning, Behavior and Posture, Clarity of information and Emotional dimension). Two focus groups and a development group, composed by faculty members and standardized patients experienced in assessment, were responsible for creating the scale. The CAS was tested on 332 students from the 3rd and 6th year of medical school, with a total of 2754 assessments, performed by both faculty members and standardized patients. A descriptive analysis was conducted, as well as an Exploratory Factor Analysis, a Confirmatory Factor Analysis and a Cronbach’s alpha analysis to establish internal reliability. The final Minho-CAS, composed by 19 items, proved to be simple to use and to have very good psychometric properties.
The Minho-CAS captures five important dimensions of communication skills, with good psychometric properties. Our results show that the Minho-CAS is a valid scale to assess communication skills and can be accurately replicated on other Objective Structured Clinical Examination (OSCEs) exams focusing on communication abilities.
Leaflet making as part of module assessment in Paediatrics and Child Health intercalated Bachelor of Sciences Degree (iBSC)
* Ain Satar, Paul Winyard, Caroline Fertleman
* Corresponding author: Ain Satar, Institute of Child Health, United Kingdom
The Institute of Child Health (ICH) runs the first iBSC programme in Paediatrics and Child Health in the world, currently in its ninth year. University College London (UCL) students spend six years in medical school including a compulsory year on iBSc programme. The programme has five modules, each with a variety of assessment approaches. The Child and Community module is partly assessed through a patient/parent information leaflet designed by the students.
Summary of Work
The leaflet is submitted alongside a supportive essay justifying their work. Paediatricians and parent markers assess the leaflets on a criterion based mark scheme. The assessment aims to engage students’ higher level thinking skills; encouraging analysis, evaluation and creation. We focus on the application of knowledge, justification of actions and ultimately the development of original work (Anderson, Krathwohl et al, 2001). Although this innovative assessment has many potential benefits, we risk challenging the students beyond their ‘zone of proximal development’ (Vygotsky, 1980). We analysed leaflet marks in comparison to similar pre-seen assignments completed during the year in an effort to modulate and improve its overall utility.
Summary of Results
Comparing to other pre-seen assessments, leaflet marks were consistently lower; 17 (85%) students obtained worse marks for leaflet than the mean of three similar assessments (mean difference 6.9%, range 0.3–17.7%). The nearest directly comparable assessment is developing a clinical guideline and when compared to this, the students still scored lower 61% vs 67%. Student feedback highlighted need for more guidance on the principles of leaflet making. Although discovery learning has potential for significant impact on transformative practice, it has to be acknowledged that guided instructions are paramount in fulfilling learning objectives (Alfieri, 2011). In 2018/19, we focussed on leaflet design and construction which was well received (76% rated it 4 or 5 out of 5 with 65% of students completing feedback overall) and resulted in improved performance; thirteen (65%) students performed better compared to mean of three similar assessments (mean difference 2.3%, range 0.3–11.7%). Mean marks for leaflet and guideline assessment were similar in 2018/19 (67% vs 66%).
Discussion and Conclusion
Creative design, critical appraisal, conceptualisation and effective communication are skills frequently honed in the years following medical school. Early introduction through assessments such as leaflet design encourages development of a holistic medical professional committed to quality improvement. Medical schools should be preparing future doctors who are able to adapt rapidly to unfamiliar situations. Challenging medical students with novel assessments may increase adaptability as clinicians. However, it must be recognised that students may perform less well in novel assessments given that it is quite disparate from other aspects of medical school assessment. We have found that slight changes in teaching strategy can transform onerous assessments into an enriching learning opportunity.
Take home messages
Leaflet design as an example of a novel method of assessment in undergraduate medical education. Continued review of performance and feedback help tailor programme development and maximise learning.
Objective Structured Knowledge Assessment (OSKA) - a novel formative assessment tool
Adam T Misky, Karim Meeran, * Amir H Sam
* Corresponding author: Amir Sam, Imperial College School of Medicine & Medical Education Research Unit, United Kingdom
The importance of asking effective questions to enhance learning in medicine has been recognised in educational literature. (1) The ability to articulate responses to clinical questions is a vital skill in the clinical environment. Increasing clinical service pressures have led to a decline in the number of encounters where students are exposed to this style of learning. (2) Assessment of medical students’ ability to articulate their responses to clinical questions, as opposed to recalling factual information, is suboptimal in undergraduate medical examinations. We developed a formative assessment tool, the Objective Structured Knowledge Assessment (OSKA), where students are questioned on specific clinical topics and scenarios, and encouraged to articulate their responses.
Summary of work
To evaluate student satisfaction, students who attended the OSKA were asked to complete an anonymous online questionnaire of 13 questions relating to student satisfaction, clinical and curricular relevance, and feedback compared to traditional clinical teaching, immediately after the OSKA. Answers were collected on a five-point Likert scale. We also examined the correlation between the students’ performance in the OSKA, and single best answer (SBA) questions assessing the same clinical topics.
Summary of results
214 students took the OSKA, of which 125 students answered the questionnaire. 97.6% of students were very satisfied or satisfied with the quality of the session. 99% of students thought the topics discussed were clinically relevant. 79.2% of students felt more at ease and 84.8% felt more engaged by the session than available conventional clinical teaching. 82.4% of students felt the quality, and 85.6% of students felt the quantity of feedback received was better than feedback offered at available conventional clinical teaching sessions. Pearson correlation coefficient between the students’ OSKA scores and SBA scores was 0.233 (p
Clinical Prioritisation Questions: a novel assessment tool to encourage tolerance of uncertainty?
* Amir H Sam, Rebecca K Wilson, Martin Lupton, Colin R Melville, Omid Halse, Joanne Harris, Karim Meeran
* Corresponding author: Amir Sam, Imperial College London, United Kingdom
Uncertainty is a common problem in clinical practice. Our current summative assessments commonly ask students for the single best answer. These assessments may give students a misleading view as to the level of certainty in clinical practice. We therefore need to modify our assessments to emphasise clinical reasoning and introduce the possibility of more than one correct answer.
Summary of work
We have developed clinical prioritisation questions (CPQs), an assessment format in which students prioritise possible answers in order of likelihood. This tool was piloted with a group of medical students and evaluated in comparison to the traditional single best answer (SBAs) question format in a team-based learning setting. During the teaching session, students were asked to complete a survey regarding their feelings around uncertainty in medicine and their perceptions of CPQs and SBAs.
Summary of results
A total of 245 students participated in the teaching session. Students felt significantly more uncertain and anxious when faced with CPQs and reported that they felt ongoing use would help improve their tolerance of uncertainty (p