1Department of Surgery, School of Medicine, Kangwon National University, Chuncheon, Korea. 2Department of Pediatrics, School of Medicine, Kangwon National University, Chuncheon, Korea. 3Department of Obstetrics & Gynecology, School of Medicine, Kangwon National University, Chuncheon, Korea. 4Department of Urology, School of Medicine, Kangwon National University, Chuncheon, Korea.
Corresponding author:
Ja-Kyoung Kim, Tel: 033-258-2316, Fax: 033-258-2418, Email: kjaky@kangwon.ac.kr
Received: 25 October 2008 • Accepted: 16 January 2009
PURPOSE We developed an objective structured oral examination (OSOE) case to assess the medical ethics of students. The aim of this study was to assess the reliability of OSOE with generalizability theory.
METHODS One 10-minute OSOE that contained key questions was developed. The evaluation sheet consisted of 4 domains: moral sensitivity, moral reasoning, decision making, and attitude. The total number of items was 13. The numbers of checklist items and global rating items were 11 and 2, respectively. Items and key questions were validated by 6 professionals. Standardization of the raters and the pilot study was performed before the OSOE. Fifty-four third-year medical students participated in the OSOE. The OSOE was duplicated, and 2 professors assessed 1 student independently. Each station lasted 8 minutes and was followed by a 2-minute interval,during which raters completed the checklist forms. We analyzed the reliability of the OSOE with the GENOVA program.
RESULTS The reliability (generalizability coefficient) was 0.945, and the interrater agreement was 0.867. The type of item, checklist or global rating, was the largest variance component. The reliability of the checklist alone was 0.668 and that of the global rating alone was 0.363.
CONCLUSION The OSOE is reliable and can be used to assess ethics. More research should focus on achieving validity.
Structured viva validity, reliability, and acceptability as an assessment tool in health professions education: a systematic review and meta-analysis Abdelhamid Ibrahim Hassan Abuzied, Wisal Omer Mohamed Nabag BMC Medical Education.2023;[Epub] CrossRef
Medical ethics education in the medical school curriculum Jang Han Kim Journal of the Korean Medical Association.2017; 60(1): 18. CrossRef
The School Effect on the Reliability of Clinical Performance Examination in Medical Schools Mi Kyoung Yim, Gue Min Lee Korean Journal of Medical Education.2010; 22(3): 215. CrossRef
