Poster presentations
Chair: Iain Robinson, Co-chair: Kevin Brandom | 9th March 2021, 12:30 -15:30 (UK’s time)
Application of the Kane’s Validity Framework for Assessments in Medical Education - A Systematic Review
* Mostafa Dehghani Poudeh, Aeen Mohammadi, Shadi Asadzandi, Rita Mojtahedzadeh
* Corresponding Author: Mostafa Dehghani Poudeh, Tehran University of Medical Sciences (Iran), mft2084@gmail.com
Introduction
Despite a long time passed from the first tossing of the argument validity by Kane, the studies that have applied this model have not been explored regarding the types of the evidences used. This study inspects the most prevailing facts, used for evaluating the validity of student assessments.
Methods
We systematically reviewed the Web of Sciences, Scopus, Embase, Science Direct and Ovid for validation studies targeting the assessment methods and instruments, used for under graduate and graduate learners in medical sciences. We investigated the English language articles, both review and original ones, that only evaluated the validity of their exams before moving to practice. Only studies that used Kane’s validity framework were included in our study. We then evaluated the quality of the final articles by means of QUESTS criteria.
Findings
Four articles finally were extracted from a bulk of 3450 cases in a systematic way. All of the final articles were at good quality levels in all six QUESTS indices and were eligible for inclusion in our review study. There were fair varieties of evidences, used for exploring the four levels. However, the four levels were not evidenced with similar attentions. Extrapolation and implication were respectively the most and the least preferred levels in validation studies. We also pointed to some restrictions these studies had faced in providing the presumed evidences.
Discussion
No single evidence or even method fits for all Kane’s validation enterprises. Moreover, both quantitative and qualitative evidences should be used in such a framework.
Conclusion
It is suggested for the Kane’s validity studies to assume realistic and available evidences at the early phases of the assessment developments. The future studies might be focused on all four levels of the Kane’s framework, particularly the implication.