PURPOSE
The third year students at Gyeong-Sang National University College of Medicine were asked to perform
objective
structured clinical examination (OSCE) at the end of the 2004 academic year. In this article, we analysed the correlation among the scores of OSCE with those of Final Term Examination in Internal Medicine (FTE), Clerkship Performance in Internal Medicine (CP) and Case Conference Examination (CCE), which assess knowledge, skill and attitude, and problem solving ability in medicine, respectively. We also analyzed difficulty and discrimination indexes of the test questions or evaluation criteria, and assessed their objectivity. METHODS: 1) Relevance: the pearson correlation analysis was performed on the scores of 85 students from 6 OSCE stations, FTE, CP and CCE. 2) Analysis on the evaluation criteria: the difficulty and discrimination indexes of the 59 evaluation criteria were calculated. 3) Objectivity: in order to assess objectivity in the scores, the Student t-test was performed on the scores of students from 4 OSCE stations where there was a change in the examiners after the morning sessions and on the students from 2 OSCE stations that had no change of examiners. RESULTS: 1) Relevance: Correlation coefficients between the scores of OSCE and FTE, CP or CCE were 0.335, 0.326 and 0.421, respectively (p < 0.01). 2) Analysis on the evaluation criteria: difficulty indexes of the 59 criteria ranged from 0.15 to 0.98, and 43% of the criteria belonged to the acceptable range. Discrimination indexes ranged from -0.22 to 0.61, and 69% of the criteria were acceptable. 3) Objectivity: The student t-test showed significant differences between the scores of the morning and afternoon sessions in 2 of the 4 stations, where the examiners were changed in the afternoon (p < 0.01). CONCLUSION: Our results suggest that OSCE performed in 2004 may have relevance to a valid tool in assessing clinical competence of medical students in terms of knowledge, skill, attitude and problem solving ability. However, in order to reuse the test materials in the future, evaluation criteria need to be refined further based on difficulty and discrimination indexes, and measures to improve objectivity of examiners should also be studied.
