PURPOSE The purpose of the study was to examine the recording accuracy of faculty observers and standardized patients (SPs) on a clinical performance examination (CPX).
METHODS This was a cross-sectional study of a fourth-year medical students' CPX that was held at a medical school in Seoul, Korea. The CPX consisted of 4 cases and was administered to 118 examinees, with the participation of 52 SP and 45 faculty observers. For the study we chose 15 examinees per case, and analyzed 60 student-SP encounters in total. To determine the recording accuracy level, 2 SP trainers developed an answer key for each encounter. First, we computed agreement rates (P) and kappa coefficient (K) values between the answer key-SPs and the answer key-faculty observers. Secondly, we analyzed variance (ANOVA) with repeated measures to determine whether the mean percentage of the correct checklist score differed as a function of the rater, the case, or the interaction between both factors.
RESULTS Mean P rates ranged from 0.72 to 0.86, while mean K values varied from 0.39 to 0.59. The SP checklist accuracy was higher than that of faculty observersat the level of item comparison. Results from ANOVA showed that there was no significant difference between the percentage of correct scores by the answer key, faculty observers and SPs. There was no significant interaction between rater and case factors. CONCLUSION: Acceptable levels of recording accuracy were obtained in both rater groups. SP raters can replace faculty raters in a large-scale CPX with thorough preparation.
A study on evaluator factors affecting physician-patient interaction scores in clinical performance examinations: a single medical school experience Young Soon Park, Kyung Hee Chun, Kyeong Soo Lee, Young Hwan Lee Yeungnam University Journal of Medicine.2021; 38(2): 118. CrossRef
A Comparative Study on the Assessment of the Quality of Life by Older Cancer Patients and Caregivers and Assessment of Performance Status by Medical Staff Kyoungwon Choi, Hoonsik Bae, Yeon Ok Lim, Ilsung Nam, Hyunsook Yoon, Yojin Kim, Hyen Joo Lee Korean Journal of Health Promotion.2015; 15(3): 150. CrossRef
Assessment of Clinical Skills With Standardized Patients: State of the Art Revisited David B. Swanson, Cees P.M. van der Vleuten Teaching and Learning in Medicine.2013; 25(sup1): S17. CrossRef
Teaching Clinical Performance Examination Using Action Learning Techniques Kyung Hye Park, Woo Jeong Kim Korean Journal of Medical Education.2012; 24(1): 23. CrossRef
Analysis of First Clinical Skills Examination in the Korean Medical Licensing Examination: Focus on Examinees' Experience in a Medical School Kyung Ae Jun, Sang Yop Shin Korean Journal of Medical Education.2011; 23(3): 203. CrossRef
How to Succeed in Carrying Out the First Trial of the Clinical Skills Examination in the Korean Medical Licensing Examination Sun Huh Korean Journal of Medical Education.2009; 21(3): 211. CrossRef
