Abstract
Online large-group teaching (OLGT), employed to reach a large group of learners in separate physical locations, allows asynchronous learning and facilitates social distancing. While online large-groups can be a powerful and resource-lean means of health professions education, it has challenges and potential pitfalls that may affect the learning process and outcomes. Through a sociomateriality framework, this article describes strategies for effective online large-group teaching in health professions education in three key strands. Firstly, to optimize learning, OLGT sessions should match learning needs with appropriate OLGT platforms, incorporate strategies to sustain learner attention, and accommodate learners of different abilities. Secondly, to develop a learning culture, OLGT must not only focus on cognitive aspects of learning but also build a community of practice, nurture digital professionalism and professional identity. Thirdly, we discuss the avoidance of pitfalls such as cognitive overload of both tutors and learners, technical issues and security risks, mitigating inequities in access to online learning, and the use of program evaluation to plan for sustained improvements. We conclude with a case vignette that discusses the challenges of OLGT and the application of the above strategies in a teaching scenario.
-
Key Words: E-learning, Online learning, Large-group teaching, Webinar, Sociomateriality
Introduction
Online large-group teaching (OLGT) is a teaching technique using virtual environments, involving a large number of learners. Large-group learning is characterized by an asymmetric student-teacher ratio, where sustained discussion and interaction with every learner is not possible, and teachers cannot pay individual attention to each learner’s progress [
1]. While some define a large group as 25 or more learners [
2], the group size that would generate the above characteristics may vary based on learner maturity and material complexity. OLGT sessions may be synchronous (real-time) or asynchronous (in learner’s own time). Examples include e-learning modules, video conference tutorials and webinars, the online component of flipped classrooms, and workplace process or compliance-related e-modules. These have been utilized in undergraduate, postgraduate, and interprofessional practice settings [
3].
Social distancing requirements during the coronavirus disease 2019 (COVID-19) pandemic has led to widespread use of OLGT to replace traditional large-group teaching [
4-
6]. OLGT may also be chosen to accommodate learners at different physical locations and schedules, empower asynchronous learning, and to allow more efficient use of faculty resources. On the other hand, OLGT may pose challenges, including unfamiliar teacher-learner and learner-learner interactions, and technical difficulties, potentially compromising learning effectiveness.
This article summarizes various strategies to improve the efficacy of OLGT, employing the theoretical framework of sociomateriality. Through the lens of sociomateriality, OLGT is understood not just as a unidirectional information transfer framed in terms of cognitive processing, but instead a complex participatory process characterized by an entanglement of interactions (
Fig. 1) between learners, teachers, materials (including technological platforms), and environmental factors [
7,
8]. Some interactions are positive; for instance, OLGT may enable the use of interactive elements to enhance learning. Other interactions may be negative, for example if the learner-teacher interaction is hindered by poor internet connections or a perceived degree of separation, or if learners become distracted by their immediate environments.
Match learning need with OLGT platform
Planning an OLGT session begins with an analysis of the learning need—not just what will be taught, but also how it will be taught, and when and where learners will access the material. Sessions may be stand-alone, in a series, or part of a larger educational program; depending on the circumstances, educators need to consider which parts to cover using OLGT, as well as the overall coherence of the educational program.
Next, choose a suitable OLGT modality. Educators have a wide variety of OLGT modalities and platforms to choose from (
Table 1) [
9,
10]. Important considerations include: (1) whether learning will be synchronous (live) or asynchronous (in learner’s own time), (2) the degree of interactivity desired, and (3) logistics such as availability, platform familiarity, and ease of content creation. A combination of modalities and platforms may be used to meet the learning need. Some platforms may combine multiple modalities, for example, a web conference platform may allow both live video conferencing and simultaneous text-based discussion in the chatbox.
Incorporate strategies to sustain learner attention
Sustaining learner attention has been a traditional concern in large-group teaching, and moving this online exacerbates the problem [
11]. Learners are known to skip or fast-forward through online lectures [
12]. A number of strategies may be useful to sustain learner attention.
The first strategy is to tap on the adult learner’s self-motivation and need to know [
13,
14]. This may be done by contextualizing learning with authentic clinical scenarios that learners may encounter in their own clinical practice, thus stimulating internal motivation by inviting learners to reflect on past experiences. Transformational learning may be stimulated with disorienting dilemmas [
15], for instance by raising a scenario or hypothesis that challenges existing perspectives as the lead-in to a session.
A second strategy is to adapt active and deep learning strategies to the online platform. Active learning involves participation, intentional engagement, purposeful observation, and critical reflection [
16]. Deep learning promotes integration and application of learnt material [
17]. These strategies have been well-described for live teaching, and with some creativity can be adapted to boost learning effectiveness in the OLGT setting (
Table 2) [
18-
21].
A third strategy is to use behavior engagement tools that create opportunities for learners to engage with peers and learning material, enable learner control and agency, and avoid distractions.
Table 3 provides examples of effective and ineffective strategies [
20,
22-
25].
Accommodate learners of different abilities
Catering to learners of differing ability has always been a challenge in large-group teaching. The online environment exacerbates this challenge, for instance by reducing visual feedback of learners’ body language. However, online platforms may provide unique opportunities and tools to accommodate learners of different abilities.
The first challenge is to assess learner ability. Online instruments such as asynchronous quizzes have been described for this purpose [
26]. Similarly, short quizzes at appropriate checkpoints during synchronous OLGT sessions are helpful to gauge understanding. Teachers should plan in advance how they would respond to this diagnostic information. For instance, if a question is incorrectly answered by the majority of learners, this may reflect a misconception or poorly-understood topic which the teacher may wish to clarify before moving on. Alternatively, if only a particular subset of learners consistently answers questions incorrectly, they may benefit from additional support and remediation. It is possible to design asynchronous material such that the content covered varies in response to a pre-session diagnostic quiz—for example, for an e-module on heart failure, learners identified to lack pre-requisite knowledge of cardiac physiology may begin with a refresher, while advanced learners can skip the refresher and go straight to advanced topics.
The second challenge is to design OLGT material to cater to a range of learner abilities. For example, in an interactive voice-annotated PowerPoint or app, scaffolding-inspired elements [
27] that can make complex tasks more accessible within the learner’s zone of proximal development [
28] include: (1) providing links to pre-requisite content for learners who need review; (2) introducing a big-picture approach and roadmap before going into specifics or individual cases (cognitive structuring); (3) providing worked examples before asking learners to solve similar problems (modelling [
29]); (4) moving from simpler to more complex tasks (reduction of degrees of freedom/fading [
30]); (5) having additional clues pop up should learners take longer than expected to solve a problem (hints); and (6) designing quizzes to identify misconceptions and errors, and providing detailed feedback on each incorrect option (providing feedback [
21]).
Build a community of practice
Acculturating into a community of practice is an important part of health professions education [
31]. Were OLGT to focus only on cognitive aspects of learning, one would lose key sociological aspects of health professions education—including peer learning, distributed problem solving, and transmission of tacit knowledge [
32]. It is therefore helpful to build an online community of practice.
Building a community of practice requires deliberate effort to create mutual trust, a safe space, and authentic interactions. Possible strategies include: (1) Introducing an informal ‘forming’ stage during synchronous sessions, during which participants can warm up and get to know each other personally [
33]. In an asynchronous setting, self-introductions using bios or recorded short videos may be a substitute. (2) During synchronous sessions, have learners and teachers turn on their webcams to see each other, rather than merely interact as names on a screen. Teachers should leave their webcams on throughout the session. (3) Establishing communication guidelines to avoid inflammatory messages that erode trust and the psychological safety to share freely [
33]. (4) Cultivating collaborative learner-learner interactions, such as small-group breakout discussions and team-based assignments. This is also an effective active learning and behavioral engagement strategy [
34]. (5) Having role models and mentors, such as more senior residents and faculty members, join parts of online sessions. Role modelling is an important influence on professional identity formation [
35]. The unique advantage of online sessions is that mentors can tune in remotely and only need to commit a small amount of time, for instance to facilitate breakout discussions or reflective learning exercises. Mentoring relationships can continue offline and allow learners further opportunities to embed themselves in the community of practice.
Nurture digital professionalism and professional identity
Beyond cognitive development, health professions education also encourages professional identity formation (PIF)—where learners acquire the values and identity of a healthcare professional, through educational experiences and a process of socialization [
35], and learn how to behave professionally. OLGT creates a different socialization environment and a new hidden curriculum of what constitutes professional behavior, or “digital professionalism” [
36,
37]. In an online environment, while a less overt power hierarchy can prove welcoming and positive for interaction [
38], standards of professional behavior may slip. The norms of professionalism—including obligations to patients, ethics, and respectful communication— must be made explicit and upheld even online.
Practically speaking, it is helpful to keep abreast of new varieties of unprofessional behavior online. For example, it is possible to create a looped video of oneself to fake attendance at online sessions [
39]. Teachers should identify and remediate such unprofessional behaviors, and strive to role-model good online behaviors to reinforce digital professionalism; for example, with live video, small things are noticed—including how teachers manage unexpected disruptions from clinical staff (or their own children!). Finally, to nurture PIF, it might be helpful to incorporate sessions such as guided reflection, mindful clinical practice, and resilient responses to difficult clinical interactions, into the OLGT setting. The teaching of professionalism and ethics should not be neglected even in the move to OLGT [
5].
Manage teachers’ cognitive load during synchronous OLGT sessions
Cognitive load theory (CLT) posits that each person has a finite working memory shared between intrinsic (task-essential), extraneous (non-essential), and germane (learning-related) load. It follows that germane load, and therefore learning, can be optimized by managing intrinsic and extraneous load [
40]. Beyond the education setting, CLT has implications for patient safety [
41] and provides a lens with which to analyze and optimize tasks such as patient handoffs [
42].
CLT provides a lens to view the cognitive load of a teacher [
43] during a synchronous OLGT session. Teachers may face an intrinsic load exceeding that of in-person sessions as they respond to simultaneous discussions in the audio-visual space and chat box; often multiple learners write messages without waiting in turn for responses. Additionally, high extraneous loads may arise from unfamiliar online tools (e.g., sketching with a mouse instead of a whiteboard), having to manage technical aspects while teaching (e.g., screen share, launch polls, let students in from the waiting room), and the increased effort required to gauge learner response. Should teachers face cognitive overload, teaching quality may be compromised.
Teachers must be aware of this pitfall and manage their cognitive load [
40]. Strategies to decrease extraneous load include doing a “dry run” to familiarize themselves with the software platform, having assistance with technical aspects, and providing a graphics tablet for teachers who sketch extensively. Intrinsic load can be managed by aggregating learner responses using polls instead of inviting individual responses and creating “lecture etiquette” rules to manage multiple simultaneous responses. When cognitively overloaded, teachers should focus on one task at a time, for example letting students in from a waiting room only at appropriate breaks.
Teachers should also be mindful of their learners’ cognitive load. Learners can become overwhelmed with toggling between online platforms, keeping track of login details and how to use the various platforms. Making conscious efforts to centralize access to and streamline delivery of learning material will allow learners to focus fully on what is being taught.
Facilitate a smooth and safe OLGT experience
The OLGT learning experience may be diminished by technical issues, security risks, and inequitable access to OLGT sessions.
Firstly, technical problems such as poor sound and video quality add extraneous cognitive load and distract from the learning task. Mitigation strategies must be adapted to local requirements and the platform chosen. Dry runs may be used to iron out technical issues, and “pre-flight checklists” for synchronous sessions (such as testing live audio, ensuring features such as polls are set up correctly) reduce interruptions later. It is also advantageous to have pre-specified back-up communication mechanisms for use if learners are unable to join synchronous sessions, or if a technical failure occurs, as well as develop an IT support system for rapid troubleshooting, including after-office-hours support.
Secondly, security risks may harm learners and patients. These range from the embarrassing (e.g., inadvertent webcam activation in various states of undress) to the serious, including breaches of patient confidentiality, unauthorized third parties accessing online sessions, and even malware exposure on computer systems with confidential patient information [
44]. Best practices include choosing secure platforms, using platformrecommended security features (e.g., meeting passwords, use of a waiting room in web conferences), verifying the identity of participants before allowing access to online material or sessions, and setting ground rules such as de-identifying patient information before use.
Thirdly, it is necessary to mitigate inequities in access to online learning. Some learners may lack suitable equipment, access to high-speed internet, or a conducive learning environment; these inequities may be even more pronounced in resource-poor settings [
6,
45,
46]. Measures to leave no learner behind include choosing platforms that can work on slower internet connections, supporting learners who lack suitable equipment through the provision of loaned equipment, device subsidies, or shared access at a common facility, and recording synchronous sessions for learners who are unable to attend. Where OLGT has a global audience, the scheduling of synchronous sessions should take into account learners across different time zones.
Plan for continual improvement and system-level change
The first iterations of OLGT efforts often raise unanticipated successes and challenges. Program evaluation will help to further refine and improve OLGT programs, and are best planned for at the start. Sustained improvement can be achieved by strengthening the learning ecosystem [
5]. Aspects to consider include: (1) Choose a program evaluation approach. The form of program evaluation depends on its goals [
47]. Decisionoriented program evaluation guides decisions on how and when to use OLGT, while outcomes-oriented approaches evaluate whether sessions meet defined objectives. (2) Define desired outcomes, if an outcomes-oriented approach is used. (3) Plan for data collection. A simple feedback survey is easy to put together, but the collection of higher Kirkpatrick level data (e.g., competency assessments, behavior change, patient care outcomes) requires early preparation. (4) Obtain approval from the Institutional Review Board, if there is an intention to publish.
Next, consider how to act on data gathered from program evaluation. Quality improvement frameworks, such as the Plan-Do-Study-Act model, are helpful to drive small-scale local changes and are well-established in clinical governance [
48]. More systemic issues may require the support of multiple stakeholders to resolve, and are often intimately interconnected with institutional processes, where any fix will have downstream and unintended effects.
In this situation, a systems design framework is helpful to map key inputs and processes and visualize interactions and deficiencies [
49]. One example of such a framework, an input-process-output model, is illustrated in
Fig. 2. A strategic plan can then be implemented to fill gaps by either adapting existing systems or by creating new ones.
Putting it all together: a case example
The following vignette describes an application of these principles. Due to the COVID-19 pandemic, a small-group workshop for 75 graduating medical students on ‘managing emergencies on night calls’ was converted to an OLGT session. Using a sociomateriality lens, three key challenges were identified.
Firstly, the workshop case studies were originally designed to be participative. It was felt that the interaction between students, dealing with questions raised and errors made was a key aspect of learning (i.e., emergence [
50]). This threatened to be lost during conversion to OLGT.
Secondly, OLGT necessitates prior preparation of slides and teaching visuals. As compared to using a whiteboard or flip chart during live workshop sessions, this resulted in a loss of flexibility in that the tutor could less easily scribble new teaching points or adapt visuals based on emergent themes identified during the case studies. This illustrates the sociomateriality concept that material—in this case the web platform or whiteboard—has agency to influence teaching [
50].
Thirdly, it is known that students would approach this topic with anxiety, knowing that in a few weeks they would be at the receiving end of these scenarios, only now in real-life and with real consequences. Social forces had an implication on learning behavior [
8].
To mitigate these challenges, the following strategies were employed:
(1) Match learning need with OLGT platform. To match the learning need, a synchronous, web-conference platform was chosen. An online audience response system was intermittently deployed at various junctures when additional audience response features were desired (such as live polling, displaying learner free text responses).
(2) Incorporate strategies to sustain learner attention. Real-life night-call emergency scenarios encountered by the students’ immediate seniors were presented as a disorientating dilemma. This tapped on the learners’ self-motivation, and also built confidence that they would have some tools to manage these scenarios in a few weeks. Active learning and behavioral engagement strategies were employed including use of breakout discussions, and the use of sequentially revealed case history with key decision points students had to commit to.
(3) Accommodate learners of different abilities. Short quizzes were used at various checkpoints to gauge understanding, and detailed feedback on incorrect options was provided.
(4) Manage teachers’ cognitive load. An administrative staff member managed the polling software and breakout rooms, while a second tutor was tasked to monitor and respond to live chat. This ensured that the main tutor was not cognitively overloaded.
(5) Build a community of practice and nurture professional identity. The students’ immediate seniors were invited to join the session as mentors. Time was allocated for both structured and unstructured interaction. In addition to discussing the case scenarios, the students’ seniors also shared reflections on the difficulties they encountered on night calls (such as dealing with death) and advice on how to manage these scenarios—not only in terms of clinical management, but also in terms of managing ethical dilemmas, and coping with these challenges at a personal and emotional level.
Conclusion
The fundamental principles of health professions education hold true whether the educational setting is online or in-person, synchronous or asynchronous. The lens of sociomateriality is helpful in characterizing the complex interactions between learners, teachers, environments, and devices in OLGT. Understanding these interactions is key to optimizing learning through OLGT, and avoiding its pitfalls. Additionally, it is valuable to think about OLGT beyond individual sessions. Instead, endeavor to develop a learning culture—model digital professionalism, nurture a community of practice, build systems-level infrastructure and processes, and plan for continuous improvement. When well-designed with appropriate pedagogical strategies, OLGT can be efficient, convenient and effective. This will prove useful and sustainable in a post-pandemic world.
ACKNOWLEDGMENTS
None.
ACKNOWLEDGMENTS
None.
Funding
No financial support was received for this study.
Conflicts of interest
No potential conflict of interest relevant to this article was reported.
Author contributions
JMN Fong and LPM Tsang conceptualised this work. All authors were involved in drafting and revising this article, and all authors give final approval of the version to be published.
Fig. 1.
Schematic Illustration of Online Large-Group Health Professions Education from a Sociomateriality Viewpoint
How does large group online health professions education take place? (A) Didactic model: one-way information flow from teacher to learner. (B) Interactive model: two-way interactions between teacher & learners through use of elements such as polls and quizzes. (C) Sociomateriality model: learning characterized by complex interactions between learners, teacher, materials and environments.
Fig. 2. Example of a Simplified Input-Process-Output Model
Table 1.Online Large-Group Education Modalities and Platforms
Table 1.
|
Modality (examples of platforms) |
Advantages |
Disadvantages |
|
Synchronous |
Web conference (Zoom Video Communications, Cisco WebEx, Google Meet, etc.) |
Live teacher-learner interaction; ability to organize break-out rooms for small-group discussion. |
Requires stable and fast web connection. |
|
Text-based live discussion |
Particularly useful in cultural contexts where learners are less vocal on video, but more disinhibited over text [9]. |
Discussion can go off-topic, teacher unable to control the discussion. |
|
Synchronous or asynchronous |
Multi-user digital experiences and virtual worlds (Second Life) |
Rich learner-learner interactions. Potential for ‘virtual’ experiential learning, role-play, or simulation. |
The virtual world may pose distractions. |
|
Asynchronous |
|
|
|
|
Least interactive |
Recorded lecture/video |
Familiar to most teachers; easy to create. |
Learner attention may not be sustained. |
|
Digital chalk-talk video (Khan Academy) |
Cogent presentation of words and images alongside speech; accessibility to millennial learners [10]. |
Requires careful storyboarding & custom recording process. |
|
Voice-annotated PowerPoint |
Ability to embed interactive elements (e.g., quizzes) to sustain interaction. |
More effort to create than a lecture recording. |
|
Interactive app/site (NEJM Interactive Medical Case) |
Facilitates active learning by having learners make clinical reasoning or decision commitments. |
Content creation is challenging and requires a specialized IT team. |
|
Most interactive |
Online discussion board |
Allows non-scripted learner responses and extended discussions. |
Requires constant teacher input; learners may not participate. |
Table 2.Adapting Active and Deep Learning Strategies to Online Platforms
Table 2.
|
Example of active or deep learning strategy |
Implementation on a synchronous platform |
Implementation on an asynchronous platform |
|
Elaborative interrogation: activate relevant prior knowledge to aid integration and retention of new information [18]. |
Have learners ‘warm-up’ by sharing what they already know about a topic. |
Begin material by posing questions that trigger recall or reflection on prior experience. |
|
Problem/case-based learning: drive learning and application through solving a real clinical problem [19]. |
Pose clinical problems which learners discuss in breakout rooms. |
Frame the teaching topic as a clinical encounter; conclude by providing a worked answer. |
|
Test-enhanced learning: practice retrieval to improve long-term retention [20]. |
Create quizzes using the audience poll function; set mini-tasks (e.g., ‘write out your admission orders’). |
Embed interactive questions in voice-annotated PowerPoint or interactive material. |
|
Provide timely and specific feedback and identify areas for improvement [21]. |
Results of live quizzes enable teachers to address misconceptions and tailor feedback to the learners’ understanding. |
Provide feedback on interactive questions including explanations of incorrect options. |
Table 3.Examples of Effective and Ineffective Behavioral Engagement Strategies
Table 3.
|
Synchronous |
Asynchronous |
|
Effective strategy |
- Use of spaced breaks; check understanding, and clarifying doubts at intervals [22]. |
- Interactive elements, e.g., quizzes with an explanation of answers |
|
- Breakout discussions, e.g., Buzz groups, Think-Pair-Share [22]. |
- Improving the usability of electronic material to encourage participation [24]. |
|
- Live polls and post-session quizzes [20]; many synchronous platforms have built-in audience response systems. |
- Positive feedback and incentives for participation [24]. |
|
- Gamification [23]. |
- Self-directed learning activities [22], e.g., reflections and assignments. |
|
Ineffective strategy |
- Purely didactic lecture |
- Excessive, distracting animations, or interactive gimmicks without educational value. |
|
- Engaging only a subset of learners (e.g., in an interprofessional setting addressing only one profession). |
- Redundant information, e.g., visuals explained in both audio and text [25]. |
References
- 1. Cantillon P. Teaching large groups. BMJ. 2003;326(7386):437.
- 2. UCL Arena Centre for Research-based Education. Teaching Toolkits: large group teaching. https://www.ucl.ac.uk/teaching-learning/publications/2019/aug/large-group-teaching. Published August 1, 2019. Accessed July 1, 2020
- 3. Tudor Car L, Soong A, Kyaw BM, Chua KL, Low-Beer N, Majeed A. Health professions digital education on clinical practice guidelines: a systematic review by Digital Health Education collaboration. BMC Med. 2019;17(1):139.
- 4. Hameed T, Husain M, Jain SK, Singh CB, Khan S. Online medical teaching in COVID-19 era: experience and perception of undergraduate students. Maedica (Bucur). 2020;15(4):440-444.
- 5. Lee IR, Kim HW, Lee Y, et al. Changes in undergraduate medical education due to COVID-19: a systematic review. Eur Rev Med Pharmacol Sci. 2021;25(12):4426-4434.
- 6. Sharma D, Bhaskar S. Addressing the COVID-19 burden on medical education and training: the role of telemedicine and tele-education during and beyond the pandemic. Front Public Health. 2020;8:589669.
- 7. MacLeod A, Kits O, Whelan E, et al. Sociomateriality: a theoretical framework for studying distributed medical education. Acad Med. 2015;90(11):1451-1456.
- 8. Fenwick T. Sociomateriality in medical practice and learning: attuning to what matters. Med Educ. 2014;48(1):44-52.
- 9. Tran TT. Is the learning approach of students from the Confucian heritage culture problematic? Educ Res Policy Pract. 2013;12(1):57-65.
- 10. Rana J, Besche H, Cockrill B. Twelve tips for the production of digital chalk-talk videos. Med Teach. 2017;39(6):653-659.
- 11. Zureick AH, Burk-Rafel J, Purkiss JA, Hortsch M. The interrupted learner: how distractions during live and video lectures influence learning outcomes. Anat Sci Educ. 2018;11(4):366-376.
- 12. Ozan O, Ozarslan Y. Video lecture watching behaviors of learners in online courses. Educ Media Int. 2016;53(1):27-41.
- 13. Taylor DC, Hamdy H. Adult learning theories: implications for learning and teaching in medical education: AMEE Guide No. 83. Med Teach. 2013;35(11):e1561-e1572.
- 14. Knowles MS, Holton EF, Swanson RA. The adult learner: the definitive classic in adult education and human resource development. 6th ed. Amsterdam, Netherlands: Elsevier; 2005.
- 15. Mezirow JD. Learning as transformation: critical perspectives on a theory in progress. San Francisco, USA: Jossey-Bass a Wiley Company; 2006.
- 16. Graffam B. Active learning in medical education: strategies for beginning implementation. Med Teach. 2007;29(1):38-42.
- 17. Azer SA, Guerrero AP, Walsh A. Enhancing learning approaches: practical tips for students and teachers. Med Teach. 2013;35(6):433-443.
- 18. Kahl B, Woloshyn V. Using elaborative interrogation to facilitate acquisition of factual information in cooperative learning settings: one good strategy deserves another. Appl Cogn Psychol. 1994;8(1):465-478.
- 19. Wood SJ, Woywodt A, Pugh M, Sampson I, Madhavi P. Twelve tips to revitalise problem-based learning. Med Teach. 2015;37(8):723-729.
- 20. Brame CJ, Biel R. Test-enhanced learning: the potential for testing to promote greater learning in undergraduate science courses. CBE Life Sci Educ. 2015;14(2):es4.
- 21. Hattie J, Timperley H. The power of feedback. Rev Educ Res. 2007;77(1):81-112.
- 22. Samarasekera DD, Gwee MC, Long A, Lock B. Lectures and large groups. In: Swanwick T, Forrest K, O'Brien BC, eds. Understanding Medical Education: Evidence, Theory, and Practice. 3rd ed. Chichester, UK: Wiley-Blackwell; 2019: 113-121.
- 23. Singhal S, Hough J, Cripps D. Twelve tips for incorporating gamification into medical education. MedEd-Publish. 2019;Nov. 26. [Epub]. https://doi.org/10.15694/ mep.2019.000216.1
- 24. Sun N, Rau PP, Ma L. Understanding lurkers in online communities: a literature review. Comput Hum Behav. 2014;38:110-117.
- 25. Clark RC, Mayer RE. E-learning and the science of instruction: proven guidelines for consumers and designers of multimedia learning. 4th ed. Hoboken, USA: Wiley; 2016.
- 26. Gamage SH, Ayres JR, Behrend MB, Smith EJ. Optimising Moodle quizzes for online assessments. Int J STEM Educ. 2019;6(1):27.
- 27. Van de Pol J, Volman M, Beishuizen J. Scaffolding in teacher–student interaction: a decade of research. Educ Psychol Rev. 2010;22:271-296.
- 28. Vygotsky LS. Mind in society: the development of higher psychological processes. Cambridge, USA: Harvard University Press; 1978.
- 29. Hmelo-Silver CE, Duncan RG, Chinn CA. Scaffolding and achievement in problem-based and inquiry learning: a response to Kirschner, Sweller, and Clark (2006). Educ Psychol. 2007;42(2):99-107.
- 30. Maloch B. Scaffolding student talk: one teacher’s role in literature discussion groups. Read Res Q. 2002;37(1):94-112.
- 31. Cruess RL, Cruess SR, Steinert Y. Medicine as a community of practice: implications for medical education. Acad Med. 2018;93(2):185-191.
- 32. Heiberg Engel PJ. Tacit knowledge and visual expertise in medical diagnostic reasoning: implications for medical education. Med Teach. 2008;30(7):e184-e188.
- 33. McInnerney JM, Roberts TS. Online learning: social interaction and the creation of a sense of community. J Educ Technol Soc. 2004;7(3):73-81.
- 34. Yang N, Ghislandi P, Dellantonio S. Online collaboration in a large university class supports quality teaching. Educ Technol Res Dev. 2018;66:671-691.
- 35. Cruess SR, Cruess RL. The development of professional identity. In: Swanwick T, Forrest K, O'Brien BC, eds. Understanding Medical Education: Evidence, Theory, and Practice. 3rd ed. Chichester, UK: Wiley-Blackwell; 2019: 239-254.
- 36. Ellaway RH, Coral J, Topps D, Topps M. Exploring digital professionalism. Med Teach. 2015;37(9):844-849.
- 37. Masters K, Ellaway RH, Topps D, Archibald D, Hogue RJ. Mobile technologies in medical education: AMEE guide no. 105. Med Teach. 2016;38(6):537-549.
- 38. Ellaway R, Masters K. AMEE guide 32: e-Learning in medical education Part 1: learning, teaching and assessment. Med Teach. 2008;30(5):455-473.
- 39. Cole S. People are looping videos to fake paying attention in Zoom meetings. https://www.vice.com/en/article/7kzq5x/looping-videos-to-fake-paying-attention-in-zoom-meetings. Published March 23, 2020. Accessed August 1, 2020
- 40. van Merriënboer JJ, Sweller J. Cognitive load theory in health professional education: design principles and strategies. Med Educ. 2010;44(1):85-93.
- 41. Harry E, Sweller J. Cognitive load theory and patient safety. In: Ruskin KJ, Stiegler MP, Rosenbaum SH, eds. Quality and Safety in Anesthesia and Perioperative Care. Oxford, UK: Oxford University Press; 2016: 16-21.
- 42. Young JQ, Ten Cate O, O’Sullivan PS, Irby DM. Unpacking the complexity of patient handoffs through the lens of cognitive load theory. Teach Learn Med. 2016;28(1):88-96.
- 43. Moos DC, Pitton D. Student teacher challenges: using the cognitive load theory as an explanatory lens. Teach Educ. 2014;25(2):127-141.
- 44. Sebenius A, Mehrotra K. Zoom grapples with security flaws that sour users on app. https://www.claimsjournal.com/news/national/2020/04/03/296380.htm. Published April 3, 2020. Accessed June 1, 2020
- 45. UNESCO. Startling digital divides in distance learning emerge. https://en.unesco.org/news/startling-digital-divides-distance-learning-emerge. Published April 21, 2020. Accessed July 1, 2020
- 46. Tierney WG, Corwin ZB, Ochsner A. Diversifying digital learning: online literacy and educational opportunity. Baltimore, USA: Johns Hopkins University Press; 2018.
- 47. Blanchard RD, Torbeck L, Blondeau W. AM last page: a snapshot of three common program evaluation approaches for medical education. Acad Med. 2013;88(1):146.
- 48. Sears R, Hopf F, Torres-Ayala A, Williams C, Skryzpek L. Using Plan-Do-Study-Act cycles and interdisciplinary conversations to transform introductory mathematics courses. PRIMUS. 2019;29(8):881-902.
- 49. Rojas D, Grierson L, Mylopoulos M, Trbovich P, Bagli D, Brydges R. How can systems engineering inform the methods of programme evaluation in health professions education? Med Educ. 2018;52(4):364-375.
- 50. MacLeod A, Ajjawi R. Thinking sociomaterially: why matter matters in medical education. Acad Med. 2020;95(6):851-855.
, Li Ping Marianne Tsang2,*
