| Home | E-Submission | Sitemap | Contact us |  
Korean J Med Educ > Volume 35(4); 2023 > Article
Nugraha, Melbiarta, Visuddho, Rimbun, Sakina, Herawati, Ummah, Rejeki, and Romdhoni: Hybrid learning as alternative approach to improve Indonesian medical students’ attitude towards clinical skills during COVID-19 pandemic



Hybrid learning has been chosen as an alternative method in the conduction of clinical skill lectures during coronavirus disease 2019 (COVID-19) pandemic since students need to learn skills applied in emergency situations. This study aims to analyze students’ attitudes between hybrid and online learning methods.


A non-randomized study was conducted between hybrid group (HG) and online group (OG) on three courses (intravenous [IV]-line insertion, nasogastric tube [NGT] insertion, and neonatal resuscitation). We developed a preformed validated questionnaire to assess students’ attitudes in five domains (willingness, understanding, capacity, self-efficacy, and intended behavior). The questionnaire was applied at the pre- and post-intervention.


A total of 100 participants were included (n=49 in HG, n=51 in OG). Post-course attitudes were significantly higher in online (IV-line, p=0.000; NGT, p=0.000; resuscitation, p=0.000) and hybrid (IV-line, p=0.000; NGT, p=0.000; resuscitation, p=0.000) groups compared to their pre-course. There were no significant differences in post-course attitude between groups (IV-line, p=0.072; NGT, p=0.163; resuscitation, p=0.146). Normalized-gain scores of all subjects were higher in HG (IV-line, p=0.012; NGT, p=0.085; resuscitation, p=0.033).


In conclusion, hybrid learning could be considered as a better alternative in clinical skill lectures to maximize students’ attitudes, especially during COVID-19 pandemic.


The coronavirus disease 2019 (COVID-19) pandemic has changed the landscape of human life in any aspects unprecedentedly. Since the World Health Organization has declared COVID-19 as global pandemic, new variant of COVID-19 had continued to evolve which tremendously increase new cases in many countries and make them facing a new wave of the pandemic, including Indonesia. The delta variant of COVID-19 as it spreads in 2021 was escalating the crisis, the number of cases has increased sharply in the past month with the highest death rate ever [1,2]. Indonesia, which then fully vaccinated 5.5% of its citizens, has reported an average of 919 deaths per day over the past week, making Indonesia as new epicenter of the spread in Asia [1]. This pandemic had also led to significant impacts on education system as well as learning and teaching process [3]. The Ministry of Education and Culture had instructed to conduct distant education and learn-from-home program to limit the COVID-19 spreading, causing over 60 million students in Indonesia were affected by nationwide school closures, including medical students [4,5].
It is undeniable that medical education, learning environment, and clinical training has changed completely. This pandemic had led to significant impacts on the course of basic clinical skills. A sudden transition from face-to-face (traditional method) to virtual learning was held to suppress the spread of the virus as the mortality rate increased. However, barriers that hinder the implementation of clinical skills course could not be denied, especially in low socio-economic countries such as Indonesia. Meanwhile, clinical skills learning requires hands-on training due to the complexity between science and clinical setting application. Even though current technology has been applied to optimize online learning, this issue was still a dilemma since online learning cannot covered every aspects, especially in clinical skills [6]. Thus, a method that combine online and offline course is needed as we take the online session advantage and also practice skills offline.
Clinical skills training requires students to interrelate domains of learning including cognitive, affective, and psychomotor domains [7]. The availability and accessibility of various sources in online learning help students to learn and practice by themselves. These advantages were shown by higher knowledge and skill scores compared to the traditional method [8]. However, students reported a lack of motivation and difficulty in preparing standard skills materials. Limited exposure to clinical setting simulation in virtual learning would cause lower self-confidence to perform professionally, as these skills will later be implemented in emergency situations. Finally, the ideal objective of students’ attitudes was not gained after the virtual course [6].
As the incidence of COVID-19 tends to decrease, a review of education strategies must be carried out soon. Hybrid learning has been offered as a promising alternative yet mutualistic for both students and institutions due to optimized understanding during virtual courses strengthened by face-to-face simulation courses [9]. Hybrid learning is defined as an education method in which in-person classroom experience is combined with online experiences that includes synchronous or asynchronous interactions with peers and instructors, as well as lessons and assessments can be completed online or in person [9]. The main key concept of hybrid learning is that the online resources are used to supplement face-to-face sessions. Several studies have indicated that hybrid learning may be more effective in improving students’ outcomes compared to conventional methods which in turn will make their performance professional dealing with emergency situations [10,11].
Hybrid learning offers the opportunity to provide engaging learning experiences for students by combining face-to-face instruction with online learning opportunities [9]. This approach is necessary, even when the pandemic is over, technology-assisted teaching learning will and should be used [12]. Nevertheless, hybrid learning also faces several limitations. Difficulties in using hybrid learning, bad internet network issues, and some electronic resources problems might hinder and limit the entire learning process [13]. The effectiveness of hybrid learning may depend on student characteristics, project characteristics, learning outcomes, and high dependency on experience with the internet and computer apps [11].
It is important to highlight students’ attitude during hybrid learning since it is an essential part of self-directed learning and also necessary to pay attention and evaluate the limitations of this learning method. Yet, there are some limitations to teacher-student interaction during online class and it will be very helpful when students have a strong willingness and positive attitude towards online learning. As face-to-face sessions are limited during pandemic, a positive attitude towards learning sessions would improve and enhance hybrid learning process according to theory of planned behavior (TPB) [14]. To the best of our knowledge, the evidence demonstrating the effectiveness of hybrid learning on students’ attitude is still scarce. Hence, this study aims to analyze students’ attitude between hybrid and online clinical skills training. We intend to evaluate the student reaction, i.e., willingness, understanding, and capacity to the learning session using the Kirkpatrick model. Moreover, we also investigate self-efficacy and intended behavior which are used as critical predictors of students’ attitude improvement based on TPB [15].
This study would provide evidence in the construction of better medical education learning process, curriculum, and policies. Furthermore, this could also streamline current practices and include hybrid learning to improve teaching and learning practices in academic settings. Since the research of hybrid learning on clinical skills is limited, this study has the potential to contribute to the academic knowledge base and have broad practical applications.


1. Study design

This was a non-randomized study with non-equivalent control group pre- and post-test design. It was described according to the TREND (Transparent Reporting of Evaluations with Nonrandomized Design) statement (https://www.cdc.gov/trendstatement/index.html).

2. Participants and setting

We recruited 3rd-year pre-clinical medical students who had participated in clinical skills training with no prior experience of performing any of the skills being studied. Participants who did not complete the training did not participate in the examination, did not complete the questionnaire, or refused to participate in the study were ruled out. The sample size was calculated using G*Power software ver. (Heinrich-Heine-Universität Düsseldorf, Düsseldorf, Germany; http://www.gpower.hhu.de/) as previously described by Kang [16] in 2021. Based on the two dependent sample t-test, power analysis indicated that a total sample size of 80 participants would yield statistical power (1–β) ≥0.9 with a significance level of 0.05. Considering the 20% dropout rate, the sample size was finally determined to be 100. Out of 100 participants, 49 students who completed all of the courses virtually comprised the online group (OG) group, while 51 another completed all of the clinical skills courses in hybrid method comprised the hybrid group (HG) group. The assignment of each group is voluntarily according to students’ preferences (Fig. 1).
This study was conducted at Faculty of Medicine, Universitas Airlangga and lasted for 4 weeks, from May 10th, 2021, to June 16th, 2021. The participants were divided into two groups. The HG was students who were given the hybrid course for clinical skills training, while others given the course virtually was assigned to OG. They are asked to fill out the questionnaire before and after the training to evaluate their attitude towards the given course. Prior to the intervention, a pre-test was given to both groups and after implementing the training, the participants completed a post-test.

3. Interventions

Each participant in both groups were given three courses, (1) intravenous (IV)-line insertion, (2) nasogastric tube (NGT) insertion, and (3) neonatal resuscitation. The training consists of one session (2 hours) of regular theory class and two sessions (each was 2 hours) of practice class. Each session was provided by professional and certified lecturers. For the HG, the regular theory session was held online via video conference meeting and the practice session was held offline at the campus. For the OG, all sessions were held virtually through video conference meetings. All participants were introduced to the study background, aim, procedures, and information for consent via video conference meeting. We collected data on participants’ age, gender, admission, district zone, and attitude questionnaire results using online questionnaire form (Google LLC, Mountain View, USA; docs.google.com/forms).

4. Data sources/measurement

Variables examined were student’s attitude scores before and after taking the courses and normalized-gain (N-Gain) scores of student’s attitudes from pre- to post-intervention. The attitude was measured using a validated questionnaire. We evaluated and compared these outcomes between HG and OG. We developed a typical questionnaire to assess students’ attitude. The construction of the questionnaire was carried out through comprehensive literature search and modified to fit the relevant context, validated, and presented to respondents in Bahasa Indonesia. The survey was originally written in English and translated into Bahasa Indonesia by forward and reverse translation.
The questionnaire was constructed from several attitude domains, including willingness, understanding, capacity, self-efficacy, and intended behavior (Supplements 1, 2). Willingness domain can be defined as an individual’s openness to risk opportunity or what they would be willing to do under certain circumstances [17]. Understanding is a comprehensive awareness of knowledge and the competence to apply it effectively in clinical practice [18]. Capacity refers to an individual’s cognitive ability to absorb, process, and apply medical knowledge and skills [19]. Self-efficacy pertains to assessments of one’s capability to effectively perform the necessary actions to address future situations, while intended behavior involves the motivating factors that impact a behavior, serving as indicators of individuals’ willingness to exert effort and the level of commitment they intend to invest in carrying out the behavior; the stronger the intention to perform the behavior, the more likely the behavior will be performed [20].
A native Indonesian speaker and clinical skills experts validated the questionnaire’s content. We then conducted a preliminary survey of 41 samples and statistically tested using bivariate correlation and reliability tests (Cronbach’s α=0.794) as what we have conducted in the previous study [21]. There were five statements on the questionnaire for each course that were displayed on the Likert scale (1–5). The accumulation score for each participant was derived from the summation of their responses on the Likert scale, with a maximum possible score of 45 points. Our questionnaire has also been certified for copyright by the Indonesian Ministry of Law and Human Rights (No., EC00202124788). Each answer was displayed on a Likert scale (1–5) from strongly disagree to strongly agree with the specific statement. We counted the total attitude score by adding the totals for the five questions on the questionnaire.

5. Statistical methods

For each course, the paired t-test (for normally distributed data) were used to compare the score for each attitude domain before and after the training, otherwise Wilcoxon signed-rank test will be used. We also compared the post attitude score between the HG and OG using the independent t-test (for normally distributed data) or Mann-Whitney U test (for non-normally distributed data). The N-Gain score was assessed using the formula described by previous study [22].
N-Gain Score=post test score-pre text scorebest score-per test score
This formula is constructed to assess the improvement of the participant score related to the intervention received. The score interpretation has three categories, <0.3 is considered low, 0.3–0.7 is moderate, and >0.7 is high. The N-Gain score between groups was compared using the t-test or Mann-Whitney U test based on normality distribution. Statistically significant was considered using two-sided α with a p-value <0.05. All statistical analyses were performed using the IBM SPSS software ver. 23.0 (IBM Corp., Armonk, USA).

6. Ethics statement

This study was conducted in accordance with the Declaration of Helsinki and was approved by the Health Research Ethics Committee, Faculty of Medicine, Universitas Airlangga (No., 147/EC/KEPK/FKUA/2021).


1. Participants characteristics

The characteristics of study participants in HG and OG are presented below (Table 1). A total of 100 participants, consisting of 49 participants in the HG and 51 participants in the OG were assessed in this study. Among them, female dominated in both groups (64.7% and 73.5%, respectively). They have a similar mean age of around 20 years. Data showed the majority of them came from Java and Bali (OG, 82.4%; HG, 87.8%). In the OG, most of the participants were accepted to the university through the national test (47.1%), whilst in the HG was the national invitation path (34.7%).

2. Comparison of student’s attitude between groups

Comparative analysis in both groups has shown a significant increase in overall students’ attitudes after completing the course. Subgroup analysis based on attitude domains revealed substantial changes between pre- and post-attitude scores of all courses in both groups. The HG tends to have higher overall post-attitude scores compared to the OG although the difference in total-attitude score was not significant between groups. However, several domains including understanding and intended behavior domain of IV-line insertion subject, and self-efficacy of neonatal resuscitation subject were significantly higher when compared to OG (p=0.036, p=0.018, p=0.022, respectively) (Table 2, Fig. 2AC).
The value of N-Gain score in each group was analyzed according to the course (Table 3). Among the three courses, HG gained higher scores compared to the OG. The mean score in the HG could be interpreted as high gain (>0.70), whilst in the OG was medium gain (0.30–0.70). This showed a higher increase in students’ attitudes regarding clinical skills, although significant findings were found only in IV-line insertion and neonatal resuscitation subjects (p=0.012, p=0.033, respectively).


Positive attitude towards online learning is essential to optimize online learning. Difficulty in achieving expected learning outcomes which obtained during conventional learning before pandemic due to online learning has led to a discussion to find the most appropriate learning instruction. These positive attitude on online learning will greatly influence and impact on the practical session since the clinical skills training conducted virtually. It is widely accepted and agreed that clinical skills training will not be effective if carried out virtually; hence, offline implementation must be maximized as much as possible, including when we faced the COVID-19 pandemic. At the time when this research was conducted, we were still not able to run offline skill examination, so we used another assessment method, namely in the form of measuring attitude towards clinical skills. In this recent study, we found a significant improvement in the student’s total attitude both in online and hybrid course groups. However, when analyzed separately, several domains of attitude in certain subjects were significantly higher in HGs.
Significant increases found in total attitude of both groups indicate the application of online and hybrid course method were capable to achieve learning objectives, especially when face-to-face learning is limited during COVID-19 pandemic. We added further analysis to compare the attitude between groups. Although the difference in total attitude after post-test was not significant, it was higher in hybrid learning and significant when analyzed separately in several domains. The pre-post changes as shown by N-Gain score analysis was also significantly higher in hybrid course. This result was consistent with a previous study conducted by McCutcheon et al. [23] in 2018 which compared motivation and attitude of clinical supervisee skills between nursing students who received online learning and hybrid learning. Students in HGs experience higher motivation and attitude towards clinical supervisee skills with strong relationships driven by the type of educational method [23]. Another study also showed higher self-perceived knowledge and skills among medical students that gained after taking hybrid learning compared to online learning [24]. Thus, implying hybrid learning could be more beneficial and satisfying than online learning alone in gaining preferable attitude [23,24]. Our study provides novel evidence for stakeholders in the construction of better medical education.
The willingness of medical students to do self-learning is commonly involved by their motivation and adaptability. Prior study showed that compared to offline learning, equal or higher motivation is needed to attend online learning was reported by 64.4% of participants [25]. Their deprivation on practical lessons also may be the reason for low motivation on doing self-learning during online learning. The adaptability of students is also needed for the adjustment process to online learning. This adaptation is associated with their attitude, sufficient technology skills, and sufficient devices [26].
Furthermore, understanding becomes an essential part for students to comprehend the principles and concepts of medical skills. The higher score of understanding in the HG could be explained because of the direct involvement of practice. Practice learning can increase the student’s understanding toward a motoric skill such as examination or laboratory skill [27]. Students’ understanding of the medical skill is important since it is related to their self-efficacy and intended behavior. The study by Moon and Hyun [28] in 2019 reported the higher self-efficacy score in nursing students that received hybrid resuscitation skills learning than the control group. This may increase the student’s self-efficacy and prepare them to perform medical skills in the future.
In the amidst of unprecedented effects of COVID-19, this pandemic has tremendously impacted medical education, particularly clinical skills learning. Although the teaching and learning process of clinical skills cannot be done face-to-face, it is imperative to ensure that the students still gain complete understanding and clinical skill competencies. A previous study in Saudi Arabia reported online learning is less effective and evidenced to limit student participation, highlighting the importance of direct learning experiences [29]. Hybrid learning has emerged as a novel alternative strategy to ensure the continuity of the learning process. This approach is designed to improve student preparation, involvement, understanding, and encourage students to be more student-centered active learning atmosphere. In line with a study conducted by Aladwan et al. [13] in 2018, hybrid learning has been shown to improve students’ skills and made the learning process interactive, interesting, and collaborative atmosphere which encouraged teacher-student interaction outside of the classroom and gives students more time to complete assignments. This is what ultimately increased students’ learning efficiency when they use the online platform in their spare time and could accommodate the reintegration of face-to-face sessions [10].
Hybrid learning benefited from the advancement of technology which rapidly developed during this pandemic. As a viable method of adapting and transitioning from fully online to hands-on learning opportunities, hybrid learning offers several advantages. Gong et al. [10] in 2021 revealed that during clinical skills examination, HGs showed better performance as well as their knowledge examination compared to control. This is corroborated by the findings from prior meta-analysis showing hybrid learning had significantly large, better outcomes on knowledge acquisition particularly in health profession [11]. Hybrid learning, which integrates online and face-to-face education, overcomes time and space constraints, supports teaching methods that are difficult to achieve with textbooks, and covers more students without increasing resource requirements [10,11]. Regarding medical students’ attitudes towards hybrid learning, findings from previous works indicated that students had positive attitudes about hybrid learning and the majority of them see hybrid learning is more effective, enhances student engagement and maintains effectiveness of the learning process. They believed hybrid learning could improve self-learning skills which enabled them to be more involved in learning process and participation, enhanced teacher-student interaction, and more understandable [13].
Our study has some strengths and limitations. We analyzed attitudes among participants who came from the same population using pre- and post-course assessment designs. In this study, we included participants in groups with the same batch, similar population characteristics, using the same curriculum, age, and all of them were high school leavers. This will reduce heterogeneity-related confounding factors and strengthen our analysis. We provide new information about comparison of attitudes between hybrid and online learning methods, considering studies regarding hybrid learning on clinical skills training are still lacking. However, multi-centered studies with larger samples are required to illustrate better representations of general medical students.
In conclusion, there is a significant increase in all domains of attitude between HG and OG. Even though there is no significant difference of attitude between HG and OG at the end of the study, higher gain scores are shown in the HG. Therefore, hybrid learning could be considered as an alternative to maximize student’s positive attitude in clinical skill courses.
The results of this study may be useful for the course coordinator in the preparation of medical skills training for medical students, including determining the delivery methods. It will also be beneficial for students to raise awareness on improving positive attitudes towards hybrid learning in order to gain clinical skills competencies. Long-term outcome of learning delivery methods toward clinical skills implementation should be further scrutinized to evaluate the education policy amidst COVID-19. Future larger studies with longer duration and comparison analysis between attitude score and skills assessment are warranted to confirm our findings.

Supplementary materials

Supplementary files are available from https://doi.org/10.3946/kjme.2023.274
Supplement 1.
Pre-intervention Attitude Score Questionnaire for IV-Line Insertion, NGT Insertion, and Neonatal Resuscitation Course.
Supplement 2.
Post-intervention Attitude Score Questionnaire for IV-Line Insertion, NGT Insertion, and Neonatal Resuscitation Course.


We would like to express our geatest gratitude to all the Faculty of Medicine, Universitas Airlangga students who were willing to participate in this research and all staff who always supported our study.


Conflicts of interest
No potential conflict of interest relevant to this article was reported.
Author contributions
Conceptualization: DN, RRM, VV, RR, SS, LH, FCU, PSR, ACR; data curation: DN, RRM, VV; formal analysis: DN, RRM, VV; project administration: DN; funding acquisition: DN, RR; methodology: DN, VV; visualization: RRM; writing–original draft: DN, RRM, VV; and writing–review & editing: DN, RRM, VV, RR, SS, LH, FCU, PSR, ACR.

Fig. 1.

Flowchart of the Study Participant Selection, Intervention, and Attitude Evaluation

Fig. 2.

Comparison of Pre-Course and Post-Course Total Attitude Scores in Each Subject between Online and Hybrid Groups

Data are presented as median. Significant increase in students’ attitude was found after completing the course in each subject. (A) Intravenous (IV)-line insertion (online, p=0.000; hybrid, p=0.000). (B) Nasogastric tube (NGT) insertion (online, p=0.000; hybrid, p=0.000). (C) Neonatal resuscitation (online, p=0.000; hybrid, p=0.000).
Table 1.
Demography Characteristics of Online and Hybrid Groups Participants
Characteristic Online (n=51) Hybrid (n=49)
 Male 18 (35.3) 13 (26.5)
 Female 33 (64.7) 36 (73.5)
Age (yr) 20.67±0.12 20.50±0.71
 Admission path
 National invitation 14 (27.5) 17 (34.7)
 National test 24 (47.1) 11 (22.4)
 Independent path 13 (25.5) 11 (22.4)
 International class 0 10 (20.4)
Origin of residence
 Sumatra 1 (2.0) 3 (6.1)
 Java & Bali 42 (82.4) 43 (87.8)
 Kalimantan 4 (7.8) 1 (2.0)
 Papua & others 4 (7.8) 2 (4.1)

Data are presented as number (%) or mean±standard deviation.

Table 2.
Comparative Analysis of Students’ Attitude towards Clinical Skill before and after Learning Course
Attitude domain Subject
IV-line insertion
NGT insertion
Neonatal resuscitation
Online (n=51) Hybrid (n=49) p-valuea) Online (n=51) Hybrid (n=49) p-valuea) Online (n=51) Hybrid (n=49) p-valuea)
Willingness 0.641 0.706 0.495
Pre 4.00 (4.00–5.00) 4.00 (3.00–5.00) 4.00 (4.00–5.00) 4.00 (3.00–5.00) 4.00 (4.00–5.00) 4.00 (3.00–5.00)
Post 5.00 (4.00–5.00) 5.00 (4.00–5.00) 5.00 (4.00–5.00) 5.00 (4.00–5.00) 5.00 (4.00–5.00) 5.00 (4.00–5.00)
p-valueb) 0.009* 0.000* 0.031* 0.000* 0.035* 0.000*
Understanding 0.036* 0.261 0.392
Pre 3.00 (2.00–4.00) 3.00 (2.00–4.00) 4.00 (2.00–4.00) 3.00 (2.00–4.00) 3.00 (2.00–4.00) 3.00 (2.00–4.00)
Post 5.00 (4.00–5.00) 5.00 (4.00–5.00) 5.00 (4.00–5.00) 5.00 (4.00–5.00) 4.00 (4.00–5.00) 5.00 (4.00–5.00)
p-valueb) 0.000* 0.000* 0.000* 0.000* 0.000* 0.000*
Capacity 0.100 0.448 0.389
Pre 3.00 (2.00–4.00) 2.00 (2.00–3.00) 2.00 (2.00–4.00) 3.00 (2.00–3.00) 2.00 (2.00–3.00) 2.00 (2.00–3.00)
Post 4.00 (4.00–5.00) 5.00 (4.00–5.00) 5.00 (4.00–5.00) 5.00 (4.00–5.00) 4.00 (4.00–5.00) 5.00 (4.00–5.00)
p-valueb) 0.000* 0.000* 0.000* 0.000* 0.000* 0.000*
Self-efficacy 0.167 0.091 0.022*
Pre 2.00 (2.00–3.00) 2.00 (2.00–3.00) 2.00 (2.00–3.00) 4.00 (4.00–5.00) 2.00 (2.00–3.00) 2.00 (2.00–3.00)
Post 4.00 (4.00–5.00) 4.00 (4.00–5.00) 2.00 (2.00–3.00) 5.00 (4.00–5.00) 4.00 (4.00–5.00) 4.00 (4.00–5.00)
p-valueb) 0.000* 0.000* 0.000* 0.000* 0.000* 0.000*
Intended behavior 0.018* 0.058 0.185
Pre 2.00 (1.00–3.00) 2.00 (1.00–3.00) 2.00 (1.00–3.00) 2.00 (1.00–3.00) 2.00 (1.00–3.00) 2.00 (1.00–3.00)
Post 4.00 (3.00–5.00) 4.00 (4.00–5.00) 4.00 (3.00–5.00) 4.00 (4.00–5.00) 4.00 (3.00–4.00) 4.00 (4.00–5.00)
p-valueb) 0.000* 0.000* 0.000* 0.000* 0.000* 0.000*
Total attitude 0.072 0.163 0.146
Pre 15.00 (12.00–18.00) 14.00 (11.50–16.00) 15.00 (11.00–18.00) 14.00 (12.00–17.00) 13.00 (11.00–17.00) 14.00 (11.00–16.50)
Post 21.00 (20.00–23.00) 22.00 (20.50–25.00) 22.00 (20.00–25.00) 23.00 (20.00–25.00) 21.00 (19.00–23.00) 22.00 (20.00–24.50)
p-valueb) 0.000* 0.000* 0.000* 0.000* 0.000* 0.000*

Data are presented as median (interquartile range).

IV: Intravenous, NGT: Nasogastric tube.

* p<0.05 (statistically significant).

a) p-value for comparative analysis of students’ attitudes after learning courses between groups.

b) p-value for comparative analysis of students’ attitudes before and after learning courses in each group.

Table 3.
Normalized-Gain Score of Students’ Attitudes in Online and Hybrid Groups
Subject Online (n=51) Hybrid (n=49) p-value
IV-line insertion 0.60 (0.42–0.85) 0.78 (0.58–1.00) 0.012*
NGT insertion 0.66 (0.42–1.00) 0.80 (0.56–1.00) 0.085
Neonatal resuscitation 0.64 (0.42–0.78) 0.75 (0.57–0.96) 0.033*

Data are presented as median (interquartile range).

IV: Intravenous, NGT: Nasogastric tube.

* p<0.05 (statistically significant).


1. Dyer O. COVID-19: Indonesia becomes Asia’s new pandemic epicentre as delta variant spreads. BMJ 2021;374:n1815.
crossref pmid
2. Worldometer. Indonesia: COVID: 4,133,433 cases and 136,473 deaths. https://www.worldometers.info/coronavirus/country/indonesia/. Published 2021. Accessed September 7, 2021. 2021.

3. Rose S. Medical student education in the time of COVID -19. JAMA 2020;323(21):2131-2132.
crossref pmid
4. Hermawan C, Kurrahman T. Assessment of the impact of COVID-19 social restriction policy toward urban slum population’s socio-economic condition in Indonesia. In: Hidayahtulloh MA, Jati I, Sumardani D, eds. Indonesia: Post-Pandemic Outlook: Social Perspectives. Jakarta, Indonesia: BRIN Publishing; 2022:73-96.

5. UNICEF. Indonesia: after 18 months of school closures, children must safely resume face-to-face learning as soon as possible: UNICEF/WHO. https://www.unicef.org/indonesia/press-releases/indonesia-after-18-months-school-closures-children-must-safely-resume-face-face. Published 2021. Accessed August 6, 2023.

6. Olum R, Atulinda L, Kigozi E, et al. Medical education and E-learning during COVID-19 pandemic: awareness, attitudes, preferences, and barriers among undergraduate medicine and nursing students at Makerere University, Uganda. J Med Educ Curric Dev 2020;7:2382120520973212.
crossref pmid pmc pdf
7. Sönmez V. Association of cognitive, affective, psychomotor and intuitive domains in education, Sönmez Model. Univ J Educ Res 2017;5(3):347-356.
8. Pei L, Wu H. Does online learning work better than offline learning in undergraduate medical education?: a systematic review and meta-analysis. Med Educ Online 2019;24(1):1666538.
crossref pmid pmc
9. Singh J, Steele K, Singh L. Combining the best of online and face-to-face learning: hybrid and blended learning approach for COVID-19, post vaccine, & post-pandemic world. J Educ Technol Syst 2021;50(2):140-171.
crossref pdf
10. Gong J, Ruan M, Yang W, et al. Application of blended learning approach in clinical skills to stimulate active learning attitudes and improve clinical practice among medical students. PeerJ 2021;9:e11690.
crossref pmid pmc pdf
11. Vallée A, Blacher J, Cariou A, Sorbets E. Blended learning compared to traditional learning in medical education: systematic review and meta-analysis. J Med Internet Res 2020;22(8):e16504.
crossref pmid pmc
12. Gong L, Song Y, Xu Y, et al. The use of a technology-assisted and teacher-supervised online discussion platform to promote academic progress in blended embryology courses. BMC Med Educ 2022;22(1):817.
crossref pmid pmc pdf
13. Aladwan F, Fakhouri HN, Alawamrah A, Rababah O. Students attitudes toward blended learning among students of the University of Jordan. Mod Appl Sci 2018;12(12):217-227.
crossref pdf
14. Valtonen T, Kukkonen J, Kontkanen S, Sormunen K, Dillon P, Sointu E. The impact of authentic learning experiences with ICT on pre-service teachers’ intentions to use ICT for teaching and learning. Comput Educ 2015;81:49-58.
15. Anthony Jr B, Kamaludin A, Romli A, et al. Predictors of blended learning deployment in institutions of higher learning: theory of planned behavior perspective. Int J Inf Learn Technol 2020;37(4):179-196.
16. Kang H. Sample size determination and power analysis using the G*Power software. J Educ Eval Health Prof 2021;18:17.
crossref pmid pmc pdf
17. Gibbons FX. Intention, expectation, and willingness. https://cancercontrol.cancer.gov/brp/research/constructs/intention-expectation-willingness. Published September 24, 2020.

18. Van Hoof TJ, Doyle TJ. Learning science as a potential new source of understanding and improvement for continuing education and continuing professional development. Med Teach 2018;40(9):880-885.
crossref pmid
19. Young JQ, Van Merrienboer J, Durning S, Ten Cate O. Cognitive Load Theory: implications for medical education: AMEE guide no. 86. Med Teach 2014;36(5):371-384.
crossref pmid
20. Ajzen I. The theory of planned behavior. Organ Behav Hum Decis Process 1991;50(2):179-211.
21. Visuddho V, Nugraha D, Melbiarta RR, et al. Predominant aspects of knowledge and practical skills among medical students with online learning during the COVID-19 pandemic era. Med Educ Online 2023;28(1):2182665.
crossref pmid pmc
22. Coletta VP, Steinert JJ. Why normalized gain should continue to be used in analyzing preinstruction and postinstruction scores on concept inventories. Phys Rev Phys Educ Res 2020;16(1):010108.
23. McCutcheon K, O’Halloran P, Lohan M. Online learning versus blended learning of clinical supervisee skills with pre-registration nursing students: a randomised controlled trial. Int J Nurs Stud 2018;82:30-39.
crossref pmid
24. Hsu CC, Tsai SH, Tsai PJ, et al. An adapted hybrid model for hands-on practice on disaster and military medicine education in undergraduate medical students during the COVID-19 pandemic. J Acute Med 2022;12(4):145-157.
pmid pmc
25. Puljak L, Čivljak M, Haramina A, et al. Attitudes and concerns of undergraduate university health sciences students in Croatia regarding complete switch to e-learning during COVID-19 pandemic: a survey. BMC Med Educ 2020;20(1):416.
crossref pmid pmc pdf
26. Selvaraj A, Radhin V, Ka N, Benson N, Mathew AJ. Effect of pandemic based online education on teaching and learning system. Int J Educ Dev 2021;85:102444.
crossref pmid pmc
27. Beck CW, Blumer LS. The Relationship between perceptions of instructional practices and student self-efficacy in guided-inquiry laboratory courses. CBE Life Sci Educ 2021;20(1):ar8.
crossref pmid pmc
28. Moon H, Hyun HS. Nursing students’ knowledge, attitude, self-efficacy in blended learning of cardiopulmonary resuscitation: a randomized controlled trial. BMC Med Educ 2019;19(1):414.
crossref pmid pmc pdf
29. AlQhtani A, AlSwedan N, Almulhim A, et al. Online versus classroom teaching for medical students during COVID-19: measuring effectiveness and satisfaction. BMC Med Educ 2021;21(1):452.
crossref pmid pmc pdf
Editorial Office
The Korean Society of Medical Education
(204 Yenji-Dreamvile) 10 Daehak-ro, 1-gil, Jongno-gu, Seoul 03129, Korea
Tel: +82-2-2286-1180   Fax: +82-2-747-6206
E-mail : kjme@ksmed.or.kr
About |  Browse Articles |  Current Issue |  For Authors and Reviewers
Copyright © 2024 by Korean Society of Medical Education.                 Developed in M2PI