Increased correlation coefficient between the written test score and tutors’ performance test scores after training of tutors for assessment of medical students during problem-based learning course in Malaysia
Article information
Abstract
This paper is aimed at finding if there was a change of correlation between the written test score and tutors’ performance test scores in the assessment of medical students during a problem-based learning (PBL) course in Malaysia. This is a cross-sectional observational study, conducted among 264 medical students in two groups from November 2010 to November 2012. The first group’s tutors did not receive tutor training; while the second group’s tutors were trained in the PBL process. Each group was divided into high, middle and low achievers based on their end-of-semester exam scores. PBL scores were taken which included written test scores and tutors’ performance test scores. Pearson correlation coefficient was calculated between the two kinds of scores in each group. The correlation coefficient between the written scores and tutors’ scores in group 1 was 0.099 (p<0.001) and for group 2 was 0.305 (p<0.001). The higher correlation coefficient in the group where tutors received the PBL training reinforces the importance of tutor training before their participation in the PBL course.
Introduction
The goals of problem-based learning (PBL) are to develop flexible knowledge, efficient problem solving skills, self-directed learning skills, efficient cooperation skills, and intrinsic motivation [1]. To ascertain that the goals of PBL have been achieved, we used two kinds of scores—the written test score and tutors’ performance test score—in a study at MAHSA University Faculty of Medicine, Malaysia. Those items were to assess the overall performance of the students in a PBL session. This study was aimed at finding if there was a change of correlation between the written test score and tutors’ performance test scores after the tutors were trained for the PBL course for medical students.
Case report
1. Subjects and methods
This was an observational study, conducted from November 2010 to November 2012, on two groups of year 2 medical students. Group 1 consisted of 127 students and group 2, 137 students. In group 1, tutors did not receive the training for PBL; while, in group 2, the tutors received the training. Tutors’ scores assess the students’ performance based on soft skills such as communication skills, analytical skills, and presentation skills. The written test score is comprised of the scores obtained from the mini test which is given in the last session of the PBL in the form of multiple true or false questions designed to test the knowledge gained by the students on the case discussed in the PBL session. We retrieved the PBL scores and the end-of-semester scores of students after obtaining permission from the dean of the medical school. Students of each group were also divided into low, middle, and high achievers based on the grades obtained in the end-of-semester exams. Low achievers were students who got marks less than 60%; middle achievers, between 60% and 74%; and high achievers, equal to or greater than 75%. Data were compared by analysis of variance. Pearson correlation coefficient was obtained between the two kinds of scores in each group using SPSS version 19.0 (IBM Corp., Armonk, USA).
2. Results
The written test score and the tutors’ score were the highest in the high achievers for both of the groups whereas the tutors’ score was high for the middle achievers in group 2. Scores between different achievers in both groups were found to be statistically significant except for the tutors’ score in group 1 (Table 1). The correlation coefficient between the written scores and tutors’ scores in group 1 was 0.099 (p<0.001) and for group 2 was 0.305 (p<0.001).
Mean Scores of the Problem-Based Learning Course in Medical Students of MAHSA University Faculty of Medicine, Malaysia
Conclusion
As expected, all the scores were higher in the high achievers group rather than in the middle or low achiever groups. It is anticipated that high achievers performed well in two of the aspects of the PBL, not only knowledge gained but also performance level. When we compared the written test score and tutors’ score among the different achievers, there was a statistically significant difference in both the groups, except the tutors’ score in group 1 (Table 1). We found that this result could be because the tutors in group 1 were not well informed about the scoring in the PBL session at our university.
In another study conducted on the PBL assessment, it was found that a structured assessment pattern for PBL improved student participation, because the students and the tutors knew what was expected from the session. This also helped to standardize the PBL assessment pattern [2]. A study conducted in Malaysia on the PBL assessment instrument showed not much variability in the scores between the tutors [3]. In this study, we tried to determine the correlation between two kinds of scores in both the groups of students. We found a higher correlation between two kinds of scores in group 2. This implies that assessment by the tutors in group 2 was more stable than in group 1. This could be attributed to the fact that tutors underwent a PBL facilitation training before they took the PBL session for group 2. In spite of the tutors being the same for group 1 and group 2, when they were facilitating for group 1, they had no formal training on how to facilitate and assess a PBL session. Assessments of medical students in PBL are examples of assessments that depend on consistency of the raters [4]. Therefore, just one form of assessment may be not a very reliable tool for assessing a PBL session. A need for an assessment which combines the written test score the tutors’ performance test scores is essential. The preparedness of the tutors on how to assess and facilitate a PBL session is one of the important aspects we need to look into for a PBL-based curriculum.
In conclusion, scores in group 2 are statistically significant among three types of achievers. Group 2 has a better correlation between the scores than group 1 so that assessment by the tutors in group 2 can be said to be more stable than for group 1. Since tutors for group 2 received PBL training, the importance of tutor training should be stressed in this university.
Acknowledgements
We would like to thank all the medical students who participated in the study. We would also like to thank the dean, deputy dean, faculty of medicine MAHSA University, Kuala Lumpur, Malaysia and the management for giving us the permission to conduct the study.
Notes
Funding
None.
Conflicts of interest
None.