| Home | E-Submission | Sitemap | Contact us |  
top_img
Korean J Med Educ > Volume 36(3); 2024 > Article
Song, Yu, Lee, Moon, Kim, and Lee: E-portfolio utilization in medical school clinical practice: assessing satisfaction and learning advantages

Abstract

Purpose

This study aimed to investigate the difference in satisfaction and learning benefits between e-portfolios compared to paper portfolios during clinical practice in medical schools.

Methods

Utilization of and satisfaction with e-portfolios among 40 third-year medical students in the medicine department of Ajou University School of Medicine was collected using an online survey in December 2020. The collected data were analyzed using descriptive statistics and an analysis of variance.

Results

Students perceived that e-portfolios were highly beneficial for consistently documenting activities during clinical practice, when compared to paper-based portfolios (mean±standard deviation [SD]=2.60±1.22). However, the least rated aspect was that e-portfolios require less time than paper-based portfolios (mean±SD=1.80±1.14). Additionally, among the various clinical practice courses using e-portfolios, the highest satisfaction was observed with the fewest content items in the e-portfolio.

Conclusion

To maximize the potential benefits of e-portfolios, improvements in implementation and usability are essential. Additionally, for effective utilization of e-portfolios in clinical practice, it is necessary to clearly define students’ required competencies and ultimate goals, and structure content accordingly.

Introduction

A portfolio is a learning tool that allows students to track their progress and develop competency improvement plans based on feedback received during the learning process and assignment completion. When paper-based portfolios were first introduced in the mid-1980s [1], they drew educators’ attention [2]. As digital innovation technology advanced, paper-based portfolios were replaced with electronic portfolios (e-portfolios) to address accessibility and management issues [3]. In general, web-based e-portfolios are useful tools for storing learning-related information, gathering and organizing learning evidence, and demonstrating student learning and growth over time [3,4]. An e-portfolio is more userfriendly than a paper-based portfolio and has several advantages, including a lower risk of data loss, easier access and management [5], and the convenience of sharing faculty feedback regardless of place or time, only needing an internet connection [6,7]. In medical education, e-portfolios can foster experiential learning, critical self-reflection, and self-regulation among students, while faculty members can use them to track student development and provide timely and constructive comments [8]. Consequently, e-portfolios may be a useful instrument for promoting critical reflection in the writing process, assisting learners in self-directed learning, and facilitating contact and feedback between students and instructors.
Notably, when educational goals are not clearly defined between students and teachers, e-portfolios are frequently utilized to incorporate a large amount of content into content composition [9]. Imprecise requirements of how to construct an e-portfolio may lead to students forming a negative opinion regarding extensively-written eportfolios, and spending more time on them than necessary. Therefore, previous studies have suggested that e-portfolio content criteria should be kept simple and adaptable in relation to educational aims [9,10]. However, there is a paucity of empirical studies on whether the content composition of an e-portfolio, as seen by learners using them is acceptable.
This study aimed to investigate whether there are differences in satisfaction and learning benefits in clinical practice using an e-portfolio compared with a paperbased portfolio for third-year medical students in the Department of Medicine at Ajou University School of Medicine, based on the content items included in the e-portfolio used during clinical practice.

Methods

1. The design of the e-portfolio

This study uses the Korea Association of Medical Colleges (KAMC) e-portfolio consortium’s ubiquitous portfolio (U-FOLION) in the healthcare educational technology (EDUTECH) e-portfolio system. This eportfolio was first used in clinical practice by the Ajou University School of Medicine in 2020. Internal medicine, surgery, obstetrics and gynecology, pediatrics, psychiatry, neurology, and family medicine are among the seven clinical practice courses available for third-year medical students. E-portfolios have been used in the fields of pediatrics, obstetrics and gynecology, and neurology. Each faculty member chose the e-content portfolio to match the educational aims and features of each course. Table 1 shows the topics selected after examining the characteristics of each clinical practice course.

2. Participants

An online survey was administered in December 2020, at the completion of clinical practice, to 40 third-year medical students who participated in the clinical practice course at Ajou University School of Medicine in Korea from June to December 2020. All the 40 participants completed the survey.

3. Questionnaire

The survey questions regarding the benefits and satisfaction of e-portfolio usage as perceived by medical students were as follows: To compare the utility of different portfolio media (e.g., quality of evidence, user-friendliness), we adapted several items from Driessen et al. [6] and added questions regarding the quality of the portfolio’s format and structure. These items were evaluated on a 5-point Likert scale. In this study, the Cronbach’s α for satisfaction with e-portfolio utility compared to paper-based portfolios was 0.822. In addition, six questions were designed to assess the advantages and utility of e-portfolio during clinical practice. These questions covered aspects such as understanding of the clinical practice process, self-reflection, self-directed learning abilities, appropriateness of content composition, and feedback provision. All items were rated on a 5-point Likert scale.

4. Analysis

Descriptive statistics and an analysis of variance (ANOVA) were performed using Jamovi (https://www.jamovi.org). Descriptive statistics were employed to analyze the mean and standard deviations of the satisfaction levels, while ANOVA was performed to examine how the perception of the usefulness of eportfolios and the improvement in clinical practice performance varied based on the differences in items used in e-portfolios among the pediatrics, obstetrics and gynecology, and neurology clinical practice courses. Subsequently, post-hoc Scheffé’s tests were performed for further validation.

5. Ethics statement

This study was approved by the Institutional Review Board (IRB) of Ajou University Hospital (Ethics consent no., AJIRB-SBR-SUR-21-078). Informed consent was obtained from all individual participants included in the study.

Results

The satisfaction results regarding the utility of eportfolios compared with paper-based portfolios are presented in Table 2. The results of this study indicate that students’ perceptions of the usefulness of e-portfolios are overall neutral to slightly negative. Students perceive e-portfolios as facilitating consistent and accurate documentation during clinical practice. This is reflected in the positive scores for questions in Table 2, where students rated e-portfolios more useful for keeping activity records (2.60), recording activities accurately (2.55), and interacting with instructors (2.27) compared to paper portfolios. However, a notable finding is the low score (1.80) for the time-saving aspect of e-portfolios. Students did not perceive a significant reduction in time spent compared to paper portfolios. This could be due to factors like an initial learning curve for the e-portfolio system or a design that requires extensive data entry.
The results in Table 2 present satisfaction regarding the utility of e-portfolios compared to paper-based portfolios. Overall, student perceptions of the usefulness of eportfolios range from neutral to slightly negative. However, students recognize e-portfolios’ role in facilitating consistent and accurate documentation during clinical practice. This is evident in the positive scores for certain questions (as shown in Table 2), where students rated e-portfolios higher in terms of keeping activity records (2.60), recording activities accurately (2.55), and interacting with instructors (2.27) compared to paper portfolios. Nevertheless, a noteworthy finding is the low score (1.80) regarding the time-saving aspect of eportfolios. Students did not perceive a significant reduction in time compared to paper portfolios, which could be attributed to factors such as an initial learning curve for the e-portfolio system or a design that necessitates extensive data entry.
The results of perceptions regarding the experience and benefits of using e-portfolios in each clinical practice course are presented in Table 3. Students in the neurology clinical practice course had higher scores in terms of their experience and perceived benefits of using e-portfolios than students in the pediatrics and obstetrics and gynecology courses. Specifically, statistically significant differences were observed in the following aspects: the utility of e-portfolios for reflecting on and self-reflecting on clinical practice experiences (F=3.37, p<0.05), helpfulness of feedback received from e-portfolios in enhancing learning and performance abilities (F=4.94, p<0.001), appropriateness of the number of assessment items in e-portfolios (F=3.11, p<0.05), and promptness of feedback provided by faculty on submitted assignments through e-portfolios (F=9.75, p<0.001).

Discussion

This study aimed to investigate the differences in satisfaction and learning effectiveness between eportfolios and paper-based portfolios used in clinical practice by third-year medical students. To achieve this, the study examined the utility of e-portfolios compared to paper-based portfolios in the clinical practice process, the appropriateness of evaluation items and content structure, and the feedback provided through e-portfolios.
According to the findings of this study, satisfaction with electronic portfolios among students in clinical practicum courses was not significantly higher than with paperbased portfolios. While electronic portfolios were perceived to be slightly more useful in certain areas, overall, there was a neutral or slightly dissatisfied sentiment. Particularly, there was no notable satisfaction or positive inclination towards time efficiency and general preference. These results suggest that while electronic portfolios have potential benefits, there is a need to improve implementation and usability to meet user demands. Previous research has reported that e-portfolios can demonstrate long-term usefulness through systematic record management and accessibility [5], but this study did not find a positive trend in this regard. Some prior studies have pointed out the issue that considerable time may be required during the initial use of e-portfolios [9]. Therefore, considering the results of this study targeting initial users, user education and support are essential for the introduction and utilization of e-portfolios. Additionally, continuous improvement is necessary to enhance long-term utility.
This study also identified differences in the experiences and perceptions of using e-portfolios among clinical practice courses in pediatrics, obstetrics and gynecology, and neurology. The findings indicate that the workload for students varied depending on the content composition of e-portfolios in each clinical subject. This suggests that during the initial setup phase of e-portfolios, both faculty and students lacked sufficient understanding, leading to the inclusion of unnecessary content. Neurology had the fewest e-portfolio items, obstetrics and gynecology had the most, and pediatrics had an intermediate number. Consequently, neurology students reported the highest satisfaction with e-portfolio use. According to Vance et al. [10], during the initial introduction of e-portfolios, the lack of clear objectives often results in overly comprehensive content composition. Based on these insights, to effectively utilize e-portfolios in clinical practice, it is necessary to clearly define the required competencies and ultimate goals for students, and structure the content accordingly.
This study has limitations in generalizability as it surveyed only a small number of students using eportfolios during clinical practice at a single medical school. Additionally, although survey questions were based on previous research, a specific scale to measure e-portfolio satisfaction was not developed. The analysis of the number of assignments submitted by students was not conducted, which also represents a limitation. Future research should investigate the impact of content composition, the timing and quality of feedback, and the feedback providers on the learning benefits of eportfolios, and include detailed analysis regarding the number of assignments submitted by students. Moreover, developing a validated e-portfolio satisfaction scale is necessary to systematically measure satisfaction. These efforts will help optimize e-portfolio usage and maximize their learning effects.
In conclusion, this study confirms the potential benefits of e-portfolios; however, the findings regarding initial users underscore the need for user education and support. Additionally, it has been demonstrated that continuous improvement is essential for enhancing long-term usefulness. Specifically, the importance of clearly defining students’ needs and goals and structuring content accordingly for effective utilization of e-portfolios has been emphasized. Therefore, it is concluded that improvements to meet user demands and ongoing efforts are necessary for optimizing the utilization of e-portfolios and maximizing learning outcomes.

Acknowledgments

All authors thank the students who participated in the study.

Notes

Funding
No financial support was received for this article.
Conflicts of interest
No potential conflict of interest relevant to this article was reported.
Author contributions
MRS, JHY, SKL, MRK, SYM, and JHL designed the study. JHY, JHL, and SKL developed the questionnaire. MRS was involved in literature review, data analysis, and manuscript drafting. JHY and JHL contributed to the collection and analysis of data, and critical revision of the manuscript for important intellectual content. SKL, MRK, and SYM contributed to data collection and critical revision of the manuscript for important intellectual content. All the authors have read and approved the final version of the manuscript.

Table 1.
Content Composition Items of E-portfolio by Clinical Practice Course of Medical School
Items Pediatrics Obstetrics & gynecology Neurology
Planning and reflection
 Daily practice record O O O
 Weekly practice plan and reflection O O O
 Final reflection O O O
 Clinical presentation planning and self-evaluation O O O
 Clinical skills planning and self-evaluation O O O
 Attitude assessment (Professor 360 Assessment) O O O
 Docility O O -
Clinical presentation
 Outpatient preliminary record O O O
 Inpatient medical records of inpatients O O -
 Inpatient case presentation O O O
Clinical skills
 Clinical skills (rubric evaluation) O O O
 Clinical skills (observation) O O -
 Surgical observation record O - -
Others
 Topic presentation O - -
 Journal review O O -
 English presentation O - -
Table 2.
Satisfaction with the Usefulness of the E-portfolio Compared to the Paper-Based Portfolio
No. Questions Mean±SD
Q1. Using an e-portfolio is easier to use than a paper-based portfolio. 2.23±1.14
Q2. Using an e-portfolio is more useful than a paper-based portfolio to keep a record of my activities during my clinical practice. 2.60±1.22
Q3. Using an e-portfolio takes less time than a paper-based portfolio. 1.80±1.14
Q4. Using an e-portfolio can record activities in clinical practice more accurately than a paper-based portfolio. 2.55±0.96
Q5. Compared to the paper-based portfolio, the e-portfolio makes it easier to interact with faculty/practice majors in the clinical practicum process. 2.27±1.06
Q6. Using an e-portfolio can evaluate my clinical performance more accurately than a paper-based portfolio. 2.23±0.92
Q7. Using an e-portfolio is more useful than a paper-based portfolio. 2.02±0.85
Q8. I prefer the e-portfolio format to the paper-based portfolio. 1.93±0.97

SD: Standard deviation.

Table 3.
Experience of Using E-portfolio by Clinical Practice Course and Awareness of Benefits
No. Questions Pediatrics (a) Neurology (b) Obstetrics & gynecology (c) F p-value Post-hoc (Scheffé)
Q1. The use of the e-portfolio provided help in understanding the practice goals and the training process as a whole. 2.38±0.98 2.73±1.13 2.20±1.04 2.58 0.08
Q2. The e-portfolio is useful for reflecting and reflecting on my clinical practice experience. 2.45±1.01 2.88±1.07 2.27±1.11 3.37 0.038 b>a, c
Q3. The e-portfolio improves my self-directed learning ability. 2.20±1.04 2.60±1.11 2.27±1.15 1.51 0.228
Q4. The feedback I received from the e-portfolio helped me improve my learning and performance. 2.60±0.93 3.05±1.26 2.27±1.11 4.94 0.009 b>a, c
Q5. The number of e-portfolio evaluation items was appropriate. 2.20±0.97 2.63±1.08 2.05±1.15 3.11 0.048 b>a, c
Q6. The faculty in charge quickly gave feedback on the e-portfolio submission task. 2.73±1.09 3.45±1.24 2.30±1.20 9.75 0.000 b>a>c

Data are presented as mean±standard deviation unless otherwise stated.

References

1. Lorenzo G, Ittelson J, Oblinger D. An overview of e-portfolios. https://library.educause.edu/-/media/files/library/2005/1/eli3001-pdf.pdf. Published July 2005. Accessed April 5 2024.

2. Gerbic P, Lewis L, Northover M. Student perspectives of eportfolios: a longitudinal study of growth and development. Paper presented at: Proceedings of the 26th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education; December 6-9, 2009; Auckland, New Zealand.

3. Lewis KO, Baker RC. The development of an electronic educational portfolio: an outline for medical education professionals. Teach Learn Med 2007;19(2):139-147.
crossref pmid
4. Buckley S, Coleman J, Davison I, et al. The educational effects of portfolios on undergraduate student learning: a Best Evidence Medical Education (BEME) systematic review. BEME guide no. 11. Med Teach 2009;31(4):282-298.
crossref pmid
5. Joyes G, Gray L, Hartnell-Young E. Effective practice with e-portfolios: how can the UK experience inform implementation? Australas J Educ Technol 2010;26(1):15-27.
crossref pdf
6. Driessen EW, Muijtjens AM, van Tartwijk J, van der Vleuten CP. Web- or paper-based portfolios: is there a difference? Med Educ 2007;41(11):1067-1073.
crossref pmid
7. Van Tartwijk J, Driessen EW. Portfolios for assessment and learning: AMEE guide no. 45. Med Teach 2009;31(9):790-801.
crossref pmid
8. van der Schaaf M, Donkers J, Slof B, et al. Improving workplace-based assessment and feedback by an Eportfolio enhanced with learning analytics. Educ Technol Res Dev 2017;65:359-380.
crossref pdf
9. Elshami WE, Abuzaid MM, Guraya SS, David LR. Acceptability and potential impacts of innovative Eportfolios implemented in E-learning systems for clinical training. J Taibah Univ Med Sci 2018;13(6):521-527.
crossref pmid pmc
10. Vance GH, Burford B, Shapiro E, Price R. Longitudinal evaluation of a pilot e-portfolio-based supervision programme for final year medical students: views of students, supervisors and new graduates. BMC Med Educ 2017;17(1):141.
crossref pmid pmc pdf
TOOLS
PDF Links  PDF Links
PubReader  PubReader
ePub Link  ePub Link
XML Download  XML Download
Full text via DOI  Full text via DOI
Download Citation  Download Citation
  Print
Share:      
METRICS
0
Crossref
0
Scopus
396
View
26
Download
Editorial Office
The Korean Society of Medical Education
(204 Yenji-Dreamvile) 10 Daehak-ro, 1-gil, Jongno-gu, Seoul 03129, Korea
Tel: +82-32-458-2636   Fax: +82-32-458-2529
E-mail : kjme@ksmed.or.kr
About |  Browse Articles |  Current Issue |  For Authors and Reviewers
Copyright © 2024 by Korean Society of Medical Education.                 Developed in M2PI