The outcome evaluation between two different curriculum: students’ retention capacity based on comparative study in Indonesia

Article information

Korean J Med Educ. 2019;31(4):357-362

Publication date (electronic) : 2019 November 29

doi : https://doi.org/10.3946/kjme.2019.145

Jovian Philip Swatan^,1, Fundhy Sinar Ikrar Prihatanto^2,3, Nancy Margarita Rehatta^2,4,5, Atika⁶

¹Faculty of Medicine, Universitas Airlangga, Surabaya, Indonesia

²Medical Education, Research, and Staff Development Unit, Universitas Airlangga, Surabaya, Indonesia

³Departement of Anatomy, Faculty of Medicine, Universitas Airlangga, Surabaya, Indonesia

⁴Department of Anesthesiology, Faculty of Medicine, Universitas Airlangga, Surabaya, Indonesia

⁵Department of Anesthesiology, Dr. Soetomo General Hospital, Faculty of Medicine, Universitas Airlangga, Surabaya, Indonesia

⁶Department of Public Health and Preventive Medicine, Faculty of Medicine, Universitas Airlangga, Surabaya, Indonesia

Corresponding Author: Jovian Philip Swatan (https://orcid.org/0000-0003-0060-7374) Faculty of Medicine Universitas Airlangga, Prof. Dr. Moestopo Number 47, Surabaya, East Java, Indonesia, 60132 Tel: +62.81285001808 Fax: +62.315022472 email: jovianphilips@gmail.com

Received 2019 March 14; Revised 2019 May 1; Accepted 2019 September 27.

Abstract

Introduction

Medicine is a vast and continuously evolving knowledge with a concise medical curriculum required to support the learning process of medical students. In Indonesia, an undergraduate medical student must undergo a minimum of 11-semester course in medical sciences, consisted of seven-semester preclinical and four-semester clinical study. During the preclinical learning, there are two different teaching methods, which are, the old and new curriculum. The old curriculum consisted of full semester courses in conjunction with several integrated module blocks. The new curriculum consisted of clustered courses (several inter-correlating courses delivered in the allocated time) and organ-based blocks [1].

Majority of medical schools in Indonesia also follow the new curriculum due to the change of regulations by Indonesian Medical Council with Association of Indonesian Medical School in 2012, which requires both horizontal and vertical integration in at least 50% of the its curriculum [1]. It is argued that the recent changes in the medical curriculum are made to enhance students’ understanding the courses. However, there are no evidence to support this claim. Moreover, there are no evaluation studies to proof that the new curriculum gives a better outcome than the old one.

Curriculum evaluation is a necessary and important aspect of any educational system because it provides the basis for its policy decisions, for feedback on continuous adjustments and processes of its implementation. Student assessment is an important aspect of curriculum evaluation because it measures the learning quality and fundamental success of any student [2]. One of the parameters that could be used to evaluate the students learning quality is retention capacity. It reflects on the long-term memory of the student, which results in the understanding of course materials. It differs from the short term memory achieved through short term memorization and easily lost when left unused or replaced by a piece of new information [3,4].

Considering that there was no evaluation study of the old and new curriculum. This study, therefore, aims to determine which curriculum type has better outcome in terms of clinical pathology retention capacity.

Methods

This was a cross-sectional study conducted in December 2017 in one of the public universities in Indonesia. Subjects were undergraduate students at the faculty of medicine who took part in the clinical pathology course in 2017. There were two different groups in this study. The first group consisted of all 3rd year medical students who was taught using the old curriculum and willing to participate in this study. The second group consisted of all 2nd year who were taught using the new curriculum and willing to participate in this study. Exclusion criteria for this study were: (1) students who retake the clinical pathology course, (2) students that did not attend the clinical pathology examination, (3) student that did not attend the retention test, (4) students that was found cheating during the retention test, and (5) students that did not complete the retention test.

The first group took the clinical pathology course for 14 weeks (February–June 2017), parallel with seven other courses (microbiology, parasitology, philosophy, integrated module of ethics and medical law, integrated module of tropical medicine, research, and medical skills 4). The amount of credits given for completing this course is four credits and course assessment is conducted 3 times, two written examinations consisting of multiple-choice questions–single best answers (MCQ-SBA) with five options and one practical examination.

The second group took the clinical pathology course for 6 weeks (May–June 2017), along with one other course (anatomical pathology). Three credits are given upon completion and course assessment is conducted 2 times, which are, one written examination of MCQ-SBA with five options in it and one practical examination.

Results

A total of 438 students (234 3rd and 204 2nd year medical students) were enrolled in the clinical pathology course in 2017. From all its students, only 288 (146 3rd and 142 2nd year medical students) were willing to participate in this study. Eight students from the first group were excluded because the students did not complete the retention test. Final samples for this study were 138 students from the first group (49 males and 89 females) and 142 students from the second group (53 males and 89 females).

A total of 280 responses were collected and validated. Mean difficulty index (P) was 39.39±16.91 while mean discrimination index (D) was 0.34±0.14. The data showed a wide distribution in both difficulty index (5.26 to 75.66) and discrimination index (0.08 to 0.63). None of the items were left unanswered by the students.

Based on difficulty index, items were categorized into three groups (difficult P <30%, acceptable P 30%–70%, and easy P >70%). In this test, there were seven difficult items, 16 acceptable items, and two easy items.

Based on discrimination index, items were categorized into four groups (excellent discrimination D >0.40, good discrimination D 0.30–0.39, acceptable discrimination D 0.20–0.29, and poor discrimination D 0.00–0.19). There were eight items with excellent discrimination index, seven items with good discrimination index, seven items with acceptable discrimination index, and three items with poor discrimination index. There were no items with negative discrimination index.

The average of the first group (n=138) was 10.93±3.57 with the score ranging from 2 to 20. Meanwhile, the average score of the second group (n=142) was 8.56±3.19 with the score ranging from 1 to 17 (Table 1). Based on the difficulty level, the students in the first group had a higher correct answer percentage in all level of difficulties compared to the second group (p<0.05 in a difficult and acceptable level, p>0.05 in easy level) (Table 2).

Table 1.

Retention Test Score Distribution between Groups

Table 2.

Correct Answer Distribution between Groups

One sample Kolmogorov-Smirnov test for normal distribution showed that data was not normally distributed (p<0.0001). Comparison test using Mann-Whitney U-test showed a significant difference in score between the two groups (mean rank, 167.44 versus 114.32; Z-score, -5.509; p<0.0001). The reliability score using KR-20 formula was 0.62.

We also compare the percentage of decrease in students’ retention test result from the post-course written examination test result. The first group averaged at 34.02±19.74, while the second group averaged at 41.90±20.61 decrease in retention test result compared to the post-course written examination test result. Comparison test using Mann-Whitney U-test showed a significant difference between the two groups (mean rank, 124.50 versus 156.05; Z-score, -3.259; p=0.001).

Discussion

There is no study that had been conducted to compare the retention capacity between the old and new curriculum. This study found that retention capacity in the first group is higher than the second group (10.93±3.57 versus 8.56±3.19, p<0.0001). In addition, the percentage of decrease on retention test result in the first group is lower compared to the second group (34.02±19.74 versus 41.90±20.61, p=0.001). These differences might be due to: (1) amount of load and time given during courses, (2) number of parallel courses given, and (3) variations in factors that affects individual learning experience.

One might argue that the significant difference from both groups is because the discrepancy in the credit earning points. Even so, the outcome and the material given is the same. The difference in credit made the same amount of material learned in a shorter amount of time. However, when we compare the percentage of decrease on retention test result of both groups it is still significantly different. In this case, the same amount of material given in shorter time will lead to reduced retention. These findings supported the cognitive load theory which demonstrates that the brain only has a limited capacity to store information from working memory, called the cognitive load, to be further processed into long-term memory. After achieving the maximum load of information, the input received will be either forgotten or abandoned. This is where repetition plays a role. It prevents information to be either forgotten or abandoned by transforming the working memory into long-term memory [8].

Variation of courses that are given at the same time may also affect the retention of course materials. Ideally, courses material should be integrated both horizontally and vertically. This may involve lecturers from across disciplines to talk more on the same topic or organs but using different approaches based on their department or specialty. In group 1, several basic medical science courses are given during a full 14 weeks, in addition to the integrated module. This type of courses allows the student to integrate the material as shared commonalities among subjects and focused on finding the connection among topics and interest to see the whole picture of a subject [9].

Other factors that may affect a student academic performance is the student’s learning experience. It is known that learning is an individualized process which is dependent on a person’s experience. Several factors that may affect learning experience include (1) individual factors such as motivation, regularity on studying daily, and revising topics after class; (2) teacher factors such as teaching style, teacher’s guidance, and utilization of different learning instruments; (3) learning environment; and (4) other factors such as family issues and health issues [10].

Majority of the test items are categorized into acceptable difficulty level (64.00%) and high discrimination index (88.00%). In this study, we observed a wide distribution range in both difficulty index (5.26 to 75.66) and discrimination index (0.08 to 0.63). This condition may result from random guessing due to no penalties given in wrong answers. Same observations were also reported by Pande et al. [11] in the 240 physiology MCQ-SBA examination (difficulty index [P], 8.57 to 95.71; discrimination index [D], -0.54 to 0.8).

In conclusion, medical curriculum may have a significant impact on student’s retention capacity. In order to have better retention, we should consider the amount of time given to process a certain load of information. This study may be repeated for 1-year and 2-year post-course retention test with better preparation and higher reliability instrument. Further research using retention test in different courses may be conducted to analyze which curriculum that promotes students’ understanding of medical knowledge.

Article information Continued