1Department of Anesthesiology & Intensive Care Therapy, Sardjito General Hospital, Faculty of Medicine, Public Health and Nursing Universitas Gadjah Mada, Yogyakarta, Indonesia
2Department of Anatomy, Faculty of Medicine, Public Health and Nursing Universitas Gadjah Mada, Yogyakarta, Indonesia
3Department of Obstetrics and Gynecology, Sardjito General Hospital, Faculty of Medicine, Public Health and Nursing Universitas Gadjah Mada, Yogyakarta, Indonesia
© The Korean Society of Medical Education.
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
| Method | Advantages | Disadvantages |
|---|---|---|
| Digital (mLearning) | - Unlimited time and place | - Requires higher material preparation and platform maintenance costs |
| - Flexible so students can move at their own learning pace | - Lack of presence of instructors or tutors | |
| - Students can study, repeat material and practice many times outside class hours | - Depends on computer or smartphone, internet connection, learning system | |
| - It can give rise to distrust or cheating by students by asking other people to replace their presence during class or exams | ||
| Traditional | - Direct and face-to-face interaction between students and teachers | - Implementation depends on the presence of students and teachers at a specific place and time |
| - Students and teachers can immediately get feedback from the implementation of learning | ||
| - Teachers and students have a better connection | - The speed of learning or the material delivered is determined by the teacher (some students may feel the material is delivered too quickly, but there are also students who feel the material is delivered too slowly) | |
| - Teachers can assess students who actively participate and understand the material or students who are not interested or who do not understand the material |
| Search strategy | |
|---|---|
| Embase | ‘medical student’/exp OR ‘resident’/exp OR ‘consultation’/exp OR ‘postgraduate student’/exp OR ‘medical student*’:ab,ti OR ‘fellow*’:ab,ti OR ‘residen*’:ab,ti OR ‘trainee*’:ab,ti OR ‘consultan*’:ab,ti OR ‘postgraduate*’:ab,ti AND ‘augmented reality’/exp OR ‘augmented realit*’:ab,ti OR image*:ab,ti AND ‘skill’/exp OR skill*:ab,ti OR proficienc*:ab,ti |
| Web of Science | medical student OR medical students OR fellow OR fellowship OR fellowships OR resident OR residents OR trainee OR trainees OR consultant OR consultants OR postgraduate OR postgraduates AND augmented reality OR image OR augmented realities OR images AND skill OR skills OR proficiency OR proficiencies |
| Author (year) | Procedure | Study design | No. of subjects | Intervention | Control | Outcome | Assessment tool/parameter | Results |
|---|---|---|---|---|---|---|---|---|
| Albrecht et al. [24] (2013) | Legal medicine | Non-RCT | 6 | mARble app on mobile device | NA | Usability of system | AttrakDiff2 questionnaire based on Hassenzahl’s paradigm of experience design | Preliminary evaluation from Hassenzahl’s AttrakDiff2 questionnaire showed an average value of 0.381 (CI, ±0.492) for pragmatic quality. |
| Amiras et al. [25] (2021) | CT-guided intervention | Non-RCT | 16 | CT-guided intervention with AR | NA | Feasibility of system | Feedback and questionnaire | From 16 participants, 11 felt the accuracy and realism was adequate for training and 12 felt more confident about their CT biopsy skills after training session. |
| Andersen et al. [26] (2016) | Port placement + abdominal incision | Non-RCT | 20 | STAR | Conventional system | Skill improvement | Placement error, number of focus shift, time to task completion | STAR group had less placement error (45% and 68%) and fewer focus shifts (86% and 44%), but they were more slowly (19% for each task). rather than the conventional group. This means that STAR system gave less placement error and fewer focus shifts in the procedure but giving more time to complete the task. |
| Andersen et al. [4] (2019) | Surgical telementoring of cricothyroidotomy | RCT | 20 | Telementoring using a visualization of future steps–STAR | Conventional telestrator-based telementoring (Telestrator) | Skill improvement | Idle time ratio, recall error, and task performance | Trainees in the future step visualization group had 48% smaller idle ratio, 26% less recall error, and 10% higher task performance score. |
| Botden et al. [21] (2009) | Laparoscopic suturing/knot tying | Non-RCT | 18 | Performing suturing knots on suturing model on AR ProMIS ver. 2.0 | NA | Skill improvement | Time spent during knot tying and quality of knot | The result showed a statistically significant difference between the baseline knot and the knot at top of performance curve. Significant difference also found between both of assessment scores. |
| Broekema et al. [19] (2016) | Laparoscopic skill | Non-RCT | 21 | Preoperative AR instructional video | Interactive instructional video | Skill improvement | MAP consist of left and right pathlength, insertion, angular area, volume, and time | Both groups showed significant improvements from first task to fourth. After the fourth repetition, when the performance of the two groups was compared, no significant difference in improvement were found. |
| Cecilio-Fernandes et al. [5] (2020) | Transthoracic echocardiogram | RCT | 36 | AR visual feedback (HS) (n=16), combination expert and augmented feedback (EF+HS) (n=10) | EF (n=10) | Skill improvement | Time of obtaining images, image quality | On the first test EF group were faster at obtaining images. On the retention test, there were no difference in speed of obtaining images for all group, but EF+HS group had higher score on image quality than the other groups. |
| Chowriappa et al. [7] (2015) | Robot-assisted UVA | RCT | 52 | HoST AR simulator | Conventional method | Skill improvement | UVA evaluation score (needle driving, needle positioning and suture placement); GEARS score | HoST group had significant difference of evaluation score in needle driving, needle positioning and suture placement, this group also had higher GEARS score. |
| Glossop et al. [32] (2004) | Projecting preoperative plans | Non-RCT | NA | The XarTrax AR system | Conventional robotic telesurgery system | Evaluation of system | Accuracy of system on device | The system and device were recommended for carrying out precise preoperative planning and for teleconsultation applications, which allow for the effective transmission of scheduled or real-time consultations to a local caregiver with less training. |
| Heinrich et al. [20] (2021) | Laparoscopic surgery (cholecystectomy) | Non-RCT | 10 | AR interactive virtual pointer | Conventional method | Applicability and feasibility | Economy of movement, error rates, user performance | The use of interactive virtual pointer for training showed a significant improvement in economy of movement (p=0.047) and error rates (p=0.047), the user performance also got good scores (total z-score; p=0.031). |
| Ienghong et al. [14] (2021) | Emergency ultrasound knowledge | RCT | 46 | Practical ultrasound flashcards with AR and traditional POCUS learning (n=23) | Traditional POCUS learning only (n=23) | Knowledge improvement | Pretest and post-test using multiple-choice question, questionnaire | Participants in flashcard group had better performance on POCUS knowledge post-test than control group. Participants were satisfied with the flashcards. |
| Kollmann et al. [33] (2021) | Learning knobology of ultrasound equipment | Non-RCT | Students | Detection of the equipment using AR app | NA | Usability of application | Questionnaire | The app had the ability to detect specific ultrasound equipment in real environment and can be used as preparation of practical course or orientation within the lab. |
| Kratzke et al. [34] (2022) | Tumor ablation performance | Non-RCT | 15 | AR Emprint SX navigation | Standard ultrasound guidance | Evaluation of ablation (efficacy and efficiency) | Percentage under-ablation and over-ablation, time of trial, number of attempts of antenna position | 281 trials were done with the results volume decreased of under-ablations and over-ablations, less task completion time and fewer antenna placement attempts with Emprint SX. In novice group, all metrics were improved by using Emprint SX navigation. |
| Leitritz et al. [9] (2014) | Binocular indirect ophthalmoscopy | RCT | 37 | ARO (n=19) | Conventional ophthalmosco py (n=18) | Skill improvement | Ophthalmoscopy training score | Median of training score for conventional group was 1.2 and showed significant difference to ARO group. |
| Logishetty et al. [28] (2019) | Acetabular cup orientation in total hip arthroplasty | Non-RCT | 24 | AR guidance | One-on-one instruction from surgeon | Skill improvement | Accuracy of orienting cup, errors, feedback/ questionnaire of users’ experience | AR-guidance group had smaller mean errors in orientation than control group. Both groups showed skill improvement. Eleven of 12 participants said would use the AR-guidance method as training tool and 10 of 12 for procedure-specific training. |
| Mendes et al. [35] (2020) | Insertion of intravenous needles | Non-RCT | 18 | Needle insertion with PINATA | Conventional method | Skill improvement | Task completion time and number of needle insertion errors; semi-structured interviews | Correlation was found between task completion time in both methods. The differences of time and errors were found between expert and novice participants. Fourteen participants considered that PINATA can outperform the conventional system of needle insertion. |
| Menozzi et al. [10] (2020) | Ophthalmic microsurgery training | RCT | 23 | AR ITS | Natural sequence method | Skill improvement | Motor skill performance | Participants underwent training in 5 days. The result showed both training method gave improvement of motor skill. There was no significance difference in both groups, but for the first 3 days of training, the ITS group showed steeper training progress than natural sequence group. |
| Mladenovic et al. [11] (2019) | Local anesthesia of IANB | RCT | 41 | Mobile AR simulator (n=22) | Control group (n=19) | Skill improvement | Time for performing anesthesia, anesthetics success, post-clinical questionnaire | The experimental group had shorter period of time (50.0±14.3 seconds vs. 68.4±25.5 seconds) in performing IANB and higher success rate than the control group. AR group also had 90.9% anesthesia success rate, while control group was 73.7%. |
| Mladenovic et al. [36] (2022) | Local anesthesia administration | Non-RCT | 19 | E-learning training using AR dental simulator (simulated-based games) | NA | Effectiveness of simulation-based games | Validated post-training survey | All participants felt comfortable, and over 90% believed that the games could facilitate learning process and had benefit. |
| Mu et al. [29] (2020) | Ultrasound-guided PCA | Non-RCT | 30 | AR simulator for ultrasound-guided PCA | NA | Effectiveness of simulator | Face validity, content validity, performance score | The average face validity score of 4.39 and content validity score of 4.53. There was a a significant difference (p<0.05) in performances between experts and novices. Significant performance improvement was found in novice participants. |
| Muangpoon et al. [37] (2020) | Digital rectal examination | Non-RCT | 19 | AR model | Real model | Movement in examining finger is accurately aligned the virtual and real model | Questionnaire of system usefulness | Movement in examining finger were accurately aligned the virtual and real model with a frame rate of 60 fps and mean error of 3.9 mm. Participants also found that the movement was realistic (mean±SD, 3.9±1.2). |
| Müller et al. [30] (2013) | Percutaneous nephrolithotomy | Non-RCT | 53 | Mobile AR | NA | Feasibility of system | Radiation exposure, puncture time | The trainee had best performance in puncturing time by using AR, while experts did their best with fluoroscopy. Mobile AR lowered radiation exposure by a factor of 3 for trainee and 1.8 for experts. Mean visualization accuracy was 2.5 mm. |
| Nagayo et al. [27] (2021) | Suture training | Non-RCT | 2 | AR motion capture system of surgical instruments and a 3D replication system of captured procedures on the surgical field | NA | Usability of system | Tracked time of AR marker, procedure time; performance and usability of replication system | ANOVA demonstrated a significant difference among the three AR markers for both experts. Tukey-Kramer test showed significant differences (p<0.05) for all pairs of AR markers in both experts. The 3D replication system can show the entire 3D models of surgical instruments withing FOV and users can see both models and actual instruments then imitate the replicated procedure. |
| Nomura et al. [23] (2015) | Laparoscopic skills (cholecystectomy) | Non-RCT | 19 | Virtual reality simulator (LapSim) | NA | Skill improvement | Pretest and post-test using ProMIS AR | Performance score was better after LapSim training in execution time of the task (p<0.001), left and right instrument path length (p=0.001), left and right instrument economy of movement (p<0.001) than before training. |
| Oostema et al. [22] (2008) | Laparoscopic skills | Non-RCT | 46 | ProMIS AR simulator | NA | Feasibility of system | Time, path length smoothness of the procedure | The result showed a significant correlation between experience and performance in time, path length and smoothness for task 2 to 4. The strongest correlation was found for the knot-tying task (R2=0.60 for time and 0.59 smoothness). |
| Rai et al. [8] (2017) | Binocular indirect ophthalmoscopy | RCT | 28 | EyeSI AR BIO (n=13) | Conventional BIO (n=15) | Skill improvement | Raw score, total time elapsed and performance score of 3 tasks | The control group outperformed the AR simulator group in three metrics of evaluation. |
| Rochlen et al. [38] (2017) | Central line insertion | Non-RCT | 40 | First-person POVAR | NA | Usability and feasibility of first POV AR | Open ended questions and free-text responses | Participants reported that the AR system was realistic (77.5%) and the ability to view the internal anatomy was helpful (92.5%). The success rate of placing the needle was similar between experienced and non-experienced participants. |
| Ropelato et al. [17] (2020) | Micromanipulation performance in ophthalmic surgery | Non-RCT | 50 | ITS using AR system | Fixed task sequence | Skill improvement | Performance score of three repetition (before first training, before second training, after second training | A significant improvement in the first session of micromanipulation performance for both groups. For the second session, only ITS group had further improvement. |
| Shakur et al. [31] (2015) | Percutaneous trigeminal rhizotomy | Non-RCT | 92 | AR simulator for percutaneous trigeminal rhizotomy | NA | Usefulness of simulator | Post graduate year, number of fluoroscopy shots, distance from the ideal entry point, distance from ideal target (lower score is better) | The mean distance from the entry point (9.4 mm vs. 12.6 mm, p=0.01), the distance from the target (12.0 mm vs. 15.2 mm, p=0.16), and final score (31.1 vs. 37.7, p=0.02) of senior residents was lower than the junior residents. The mean number of fluoroscopy shots was similar between two groups. Increased postgraduate year level had significant result with decreased distance from ideal entry point, shorter distance from target, and better final score. |
| Sugand et al. [12] (2019) | Dynamic hip screw guidewire insertion | RCT | 45 | FluoroSim AR simulator | Conventional method | Skill improvement | Tip-apex distance, predicted cut-out rate, total procedural time, total number of radiographs, total number of guidewires retries | The training cohorts had significant improvement in all metrics. Significant difference of both groups was found in procedural time (25%), number of radiograph (57%), and number of guidewires retries (100%). After the 1-week washout period, there was also an improvement in the baseline of the learning curve, which suggests skill retention. |
| Vera et al. [6] (2014) | Fundamental of laparoscopic surgery suturing task | RCT | 19 | ART | Traditional mentoring | Skill improvement | Time and errors of the performance | ART group gained faster (b=-0.567, R2=0.92) skill improvement than the control group (b=-0.453, R2=0.74), even after 10 repetition or an hour of practice (mean 167.4 seconds vs. 242.4 seconds, p=0.014). ART group also had fewer errors (8 vs. 13). |
| Volonté et al. [16] (2011) | Laparoscopic and robotic surgery | Non-RCT | Patients who underwent laparoscopic operations | AR OsiriX | NA | Making surgical interventions easier, faster, and safer | User performance | OsiriX gave benefits to surgeon such as navigations through patients’ anatomy, high lightning regions, and marked pathologies. |
| Wagner et al. [39] (1999) | Craniomaxillofacial surgical procedure | Non-RCT | 27 | AR tele-assisted surgery method | NA | Feasibility of system | SPS | Tele-assisted surgery was applied successfully. There were found technical problems in six cases but did not interfere the system performance. |
| Wolf et al. [13] (2021) | Single ECMO cannulation training | RCT | 21 | AR instructions | Conventional instructions | Skill improvement | Training times, detailed error protocol, standardized UEQ | AR instructions was associated with slightly higher training times and lower error counts. Training times and error count did not have significant correlation. |
| Yong et al. [18] (2019) | Temporal bone surgery | Non-RCT | 7 | ODG R-7 AR Smartglasses | Conventional method | Rating of experience and user-friendliness, ability to perform/teach tasks, quality of communication | Visual analog score was used to obtain mean, SD and range. Open-ended questions were used for collecting the strength and weakness data | Both residents and instructors enjoyed being able to connect by voice, text message, or sending and receiving editable still images. Technical issues were still found and need further improvement. |
| Yudkowsky et al. [15] (2013) | Ventriculostomy catheter placement | Non-RCT | 16 | Immersive touch AR system | NA | Success rate and improvement of the procedure | Pre-test and post-test, performance score, questionnaire | The result was the success rate of simulator cannulation was increased and actual procedure outcomes showed improvement. Residents also reported that the simulations were realistic and helpful in improving their skills. |
| Zhao et al. [40] (2020) | Neonatal endotracheal intubation | Non-RCT | 45 | Capturing motions and visualization using AR HMD | NA | Accuracy of learning model | The classification accuracy of machine learning model in this study was 84.6%. |
RCT: Randomized control trial, NA: Not applicable, CI: Confidence interval, CT: Computed tomography, AR: Augmented reality, STAR: System for telementoring with AR, MAP: Motion analysis parameter, HS: Help screen, EF: Expert feedback, UVA: Urethrovesical anastomosis, HoST: Hands-on surgical training, GEARS score: Global Evaluative Assessment of Robotic Skills score, POCUS: Point-of-care ultrasound, ARO: AR ophthalmoscopy, PINATA: Pinpoint insertion of intravenous needles via AR training assistance, ITS: Intelligent tutoring system, IANB: Inferior alveolar nerve block, PCA: Percutaneous renal access, SD: Standard deviation, 3D: Three-dimensional, ANOVA: Analysis of variance, FOV: Field of view, POV: Point-of-view, ART: AR telementoring, SPS: System performance scale, ECMO: Extracorporeal membrane oxygenation, UEQ: User experience questionnaire, HMD: Head-mounted display.
