Introduction
Medical school students are currently facing a heightened emphasis on professional attributes such as attitudes, behaviors, and identity, compared with the past. The changing educational environment highlights the need for medical school applicants to demonstrate not only knowledge and skills but also a well-rounded set of attitudes [1-3]. Medical schools aim to admit candidates who excel academically while possessing the desirable qualities to become competent physicians [4].
Interview evaluations are part of the admissions process at many medical schools worldwide [5]. Various research results have been reported on the effectiveness of interview evaluations. Sladek et al. [6] established that there is a correlation between interview evaluations scores and clinical performance, and Liberty et al. [7] confirmed through his research that high interview ratings predict higher performance scores in internal medicine subinternships. Research findings suggest that medical school admission interview evaluations are susceptible to implicit biases [8] and constitute a resource-intensive process, imposing financial burdens on applicants [9].
Though limited, research on medical school admission interview evaluations in South Korea does exist. In contrast to international studies which explore a diverse array of topics including the validity, reliability, and utility of interview evaluations [10], South Korean research has primarily focused on examining candidates’ academic performance. Since 2010, a mere five articles related to interview evaluations in Korean medical schools have been indexed in academic databases such as the Korean Citation Index or Scopus. These studies, while occasionally assessing personal career values and selfassessed competencies, have all predominantly investigated the relationship between interview outcomes and academic success. There has been no research on the operational characteristics of medical school interview evaluations, the effectiveness of the interview process as a student selection method, or the perceptions of professors involved in conducting or evaluating these interviews. Therefore, this study aims to investigate the current operational status of medical school interview evaluations in South Korea, analyze their characteristics, and understand the perspectives of professors who have participated in the operation or evaluation of these interviews, focusing on the strengths, weaknesses, and areas for improvement. The research questions set for this study are as follows: First, to analyze the operational status of medical school interview evaluations in South Korea using various criteria. Second, to examine the perceptions of professors experienced in admission interview evaluations, focusing on the purposes and challenges associated with conducting these interview evaluations.
Methods
1. Study design: mixed-methods
This study employed a mixed-methods approach. The quantitative research aimed to identify overarching trends and the present state of medical school interview evaluations in South Korea by collecting and analyzing data from a survey conducted across multiple medical schools. This included information on interview evaluation procedures, the purpose of interview evaluations, and the competencies expected of medical school freshmen. Concurrently, the qualitative research aimed to explore the individual perspectives of professors who have participated in the operation or evaluation of these interview evaluations. Through focus group discussions (FGDs), insights into the operational experiences, challenges faced, and perceptions of the interview evaluation process were gathered.
2. Quantitative method and design
To collect the quantitative data, the research team concurrently employed two methods. First, through a survey, this study aimed to identify the crucial factors considered during interview evaluations at each medical school. The survey items were developed by a taskforce team (TFT) consisting of four members: three professors from two medical schools and one graduate school of medicine, and one expert in admission evaluation and management. The survey comprised a total of 17 items, including nine open-ended questions and eight closedended questions (Table 1), addressing topics such as the selection criteria used during interview evaluations, the qualities emphasized when selecting prospective students, and the methods employed for admission interview evaluations.
The survey was distributed to each medical school with the assistance of the Korean Association of Medical Colleges, and the professors responsible for admissions were asked to respond. Respondents were informed in advance that the results would be used for research purposes concerning admission interview evaluations. Survey responses were collected from February 22 to March 5, 2021, with 23 out of 39 medical schools responding, resulting in a response rate of 59.0%. Concurrently, the research team gathered data on the operational status of interview evaluations from 39 medical schools using publicly available university data. The final enrollment quotas and admission methods obtained from Higher Education in Korea (www.academy info.go.kr) were cross-checked against the survey responses provided by the medical schools. The collected data were analyzed using descriptive statistics and independent sample t-tests in IBM SPSS ver. 26.0 (IBM Corp., Armonk, USA).
3. Qualitative method and design
Qualitative data were collected to understand the perceptions of professors participating in medical school admission interview evaluations. The method used in this study was FGD which allows considerable opportunities for participant to comment, explain, and share experiences and attitudes as opposed to a structured and directive interview led by the interviewer [11]. In this study, each FGD was facilitated by two or three experts in interview evaluation, including members of the research team. For each session, two or four professors from different medical schools were invited. A total of 12 professors from 10 medical schools participated in the FGDs, which were held from August to November 2021, excluding the facilitators. Considering the coronavirus disease 2019 situation and the travel distances of the participants, four out of the five FGDs were conducted non-face-to-face (Table 2). After the professors from each medical school decided to participate in the FGD, they were provided with interview guidelines and a semi-structured questionnaire developed by the research team. The questionnaire for the FGD was developed by analyzing survey responses from professors responsible for admissions at each university, covering topics such as medical school admission procedures, the purpose and operational status of interview evaluations, challenges, and future plans for interview processes. All professors invited to participate in the FGDs had experience in planning or conducting the admissions process or interview evaluations at their respective medical schools, or had served as evaluators in interview evaluations. Before the FGDs began, participants were informed that all discussions would be recorded and transcribed for research purposes, and consent procedures were conducted. During the FGDs, facilitators guided the discussions based on the pre-provided questionnaire, gathering opinions on the current status and future plans of interview evaluations at each medical school, as well as identifying challenges and potential improvements in the interview evaluation process. If challenges or areas needing improvement were identified, solutions were discussed collaboratively with the participants. Each FGD session lasted an average of 3–4 hours, with key points recorded by the facilitators and monitoring researchers involved in the study.
Each session was recorded and transcribed, and the iterative process was repeated until topic saturation was reached. Two authors (Y.J.K. and C.W.H.), trained in qualitative research, independently coded the data related to the topic. They defined each code and created categories. The research team regularly discussed and critically reviewed the themes [12]. Multiple researchers coding the same data, assessing their agreement levels on coding, and discussing any areas of disagreement constitute a form of triangulation through the involvement of multiple researchers [13]. Throughout this process, the diverse perspectives presented by the TFT, comprising experts in medical education and admissions, contributed to the reliability of data interpretation [14]. From such a postpositivist perspective, methodological triangulation enhances the validity of a study [15].
4. Ethics statement
This study was conducted with the approval of the Institutional Review Board of Hanyang University (HYU IRB-202108-023). Prior to conducting the FGD, the researchers provided sufficient explanation to the participants regarding the research purpose, interview content, time required, and voluntary withdrawal and obtained their signed informed consent forms. All interview content was recorded with the participants’ consent.
Results
1. Quantitative data: operational status of interview evaluations in the admission process and future plan
Results of the analysis of admission procedures across South Korean medical schools are outlined below.
1) Proportion of admissions through interview evaluations by admission type
When categorizing the South Korean medical school admissions into four categories, the total number of interviewee quotas accounted for 1,275 (42.7%), relative to the overall admissions quota. Notably, for the rolling admission system, “comprehensive school records screening” had the highest participation at 703 (85.5%). In comparison, “high school grades-focused admissions” accounted for 348 (41.3%). For the “College Scholastic Ability Test (CSAT) focused admissions” with a total admissions quota of 1,205, the percentage of the interviewee quota was only 18.6% (Table 3).
2) Proportion of admissions through interview evaluations by school types, locations, and admission quota
The results of the analysis of the proportion of admissions through interview evaluations by school types (public or private), locations (metropolitan or nonmetropolitan), and admission quota (fewer than 60 students, 60 or more) for South Korean medical schools are presented in Table 4. According to the establishment type, in the rolling admission, private schools (64.2%) showed a statistically significant higher proportion of admissions through interview evaluations compared to public schools (11.1%) in the “high school grades focused admission” (p<0.01). Although not statistically significant in the “CSAT focused admissions,” overall, private schools (50.8%) tended to use interview evaluations more actively than public schools (22.1%) (p<0.05). Based on the location of medical schools, in the “CSAT-focused admissions,” medical schools located in the metropolitan area (50.0%) conducted interview evaluations more frequently than those located outside the metropolitan area (11.1%) (p<0.05). The differences based on the admission quota of medical schools were not statistically significant.
3) Future plans for interview operations
Responses were obtained from 23 medical schools (59.0%). Among them, 21 medical schools reported “conducting interview” (91.3%), one was “under discussion” (4.3%), and one reported “did not conduct interview” (4.3%). The medical schools not conducting interview evaluations cited reasons such as “difficulties in making changes owing to opposition from the university headquarters,” and “challenges in complying with the Ministry of Education’s policy on interview procedures.”
2. Qualitative data: operational experiences of professor conducting interview evaluations
1) Role of interview evaluations in medical school admission
a. Evaluating holistic competencies
Participants perceived the purpose of the admission interview as “to assess various competencies that prospective medical professionals should have, beyond intellectual capabilities, such as communication, ethics, and empathy (A-2, D-1, and G-1).” All participants agreed that the interview is a representative admissions process for assessing non-cognitive competencies.
b. Tailoring admissions to medical schools’ educational philosophies
The research participants (C-1 and J-1) stated that they prefer interview evaluations because they allow for the selection of students based on the specific philosophy and educational context of their medical schools. One participant mentioned, “Since our medical school has a philosophy of nurturing scholars specializing in research, competencies related to reasoning and designing research are considered” (I-1). Another professor emphasized, “Our university values personality, so competencies like ‘collaboration’ and ‘communication’ are included without exception in the multiple mini interview (MMI)” (C-1).
c. The dominance of “negative selection” in interview admissions
Professors, through interview experiences, empirically recognized that the “negative selection” process which filters out students who should not be admitted, is stronger than selecting students who must be admitted (positive selection). A professor stated, “There is at least one applicant every 1 or 2 years whom I think should never be admitted” (I-1).
2) Methods of conducting interview evaluations
a. Maintaining a trained interviewer pool for MMI: balancing workload and expertise
The MMI requires a significant number of evaluators. Five interviewees from three universities (A-1, A-2, B-1, B-2, and D-1) stated that they were developing their own strategies to train interview experts within their medical schools. Securing a “pool of experts trained in interview skills” was mentioned as a primary concern for each medical school. One professor complained that he was “Motivating busy professors to participate in the demanding and time-consuming admissions process is the most challenging aspect” (I-1). One medical school said “Attending ‘admission workshops’ is a prerequisite for becoming an interviewer, and we strive to ensure that new members constitute less than one-third of the interviewer team and include experienced individuals” (D-1).
b. Securing exam integrity: long-term engagement of core interviewers
At one medical school (E-1), the “core” members who participate annually as interviewers often do so for many years. The reason for choosing this method is to maintain the security of the entrance examination. In these cases, it was usually a “general person interview” rather than an MMI. “The headquarters of a university (admission office) designated an interviewer and informed the medical school, which was based on a strong security policy” (F-1).
c. Striving for objectivity amid evaluator psychological burden
The evaluators reported a high level of psychological burden during the interview assessments. One professor mentioned, “Despite receiving training multiple times as an evaluator, I made an effort not to rely on personal subjective judgment” (A-1). Another expressed, “Knowing how hard the students have prepared adds to the psychological burden” (D-1).
3. Methos of formulating interview questions
1) Ensuring expertise and fairness in item development
Several considerations were made in the development of the interview questions. For example, the participants mentioned, “We should first define the medical schools’ ‘talent selection’ we want in the students we aim to admit before developing the interview questions. However, there is an occasional error in developing questions first and then establishing medical schools’ talent for admissions selection” (H-1). They also emphasized, “Over time, creating a pattern and trend in question development should be avoided” (C-1). Additionally, they stated, “Efforts are made to prevent depletion of resources for question material” (D-1). Many medical schools operate their own educational programs and workshops for this purpose. However, active participation in evaluations requires significant time investment, and owing to low compensation, obtaining enthusiastic involvement from many professors has limitations.
2) Challenges in maintaining a consistent evaluator pool and team-based interview question development
Maintaining the evaluator pool “continuously” was not an easy task. Evaluator recruitment and training workshops often overlapped with external events, such as conferences, and capable professors were sometimes unable to participate as evaluators because of “conflicts of interest” related to their children’s admissions. In smaller medical schools, there is a shortage of personnel, especially in cases such as MMI, where a large number of evaluators are required.
On the other hand, for interview item development, it operated on a “team-based” system. Within the team, each professor or pair first created interview questions and then collectively reviewed the items developed by the team. While team-based operation was advantageous in ensuring the reliability and objectivity of the interview questions, overcoming the limitation that “It is difficult for other professors to actively intervene in questions created by another professor” (G-1) depended on the composition of the team.
3) Mitigating challenges: diverse strategies across medical schools
Medical schools have made unique efforts to overcome these challenges. One school aiming to ensure the reliability of interview questions, mentioned, “During developing items, we seek feedback from medical school students and make corrections to terminologies or sentences to enhance understanding” (A-1). Furthermore, one professor said, “To understand the qualities expected from applicants and aid in interview preparation, study meetings were conducted where research papers were reviewed” (H-1). The research participants (B-1 and B-2) stated, “To alleviate the burden on individual evaluators and ensure the reliability of assessment outcomes, we convened an ‘evaluation committee’ for further discussion when assigning an ‘F’ grade.”
Discussion
This study employed a mixed-methods approach to comprehensively explore the operational status and experiences related to medical school admission interview evaluations in South Korea. Here, we discuss three key points, emphasizing the commonalities and differences between qualitative and quantitative findings.
First, the data revealed clear patterns in the operational status of the interview evaluations across various admission types. Notably, interview evaluations were most prevalent in rolling admissions, particularly in “comprehensive school record screening” (85.5%), whereas the CSAT-focused admissions had a relatively lower rate (18.6%) concerning the number of admissions. Considering the increasing trend of introducing interview evaluations for holistic talent selection [1-3], the proportion of schools conducting interview evaluations was significantly lower than the total admission quota. These results, in conjunction with qualitative data on “negative selection,” are discussed. If interview evaluations filter out students who should not be admitted, as indicated by the interview results from professors, and if interview evaluations play a “passive role” in student selection, there may be limitations in proving the effectiveness of interview selection and expanding its operation.
In particular, the results highlight regional disparities in interview operations. Quantitative data revealed differences between private and national (and public) medical schools, and between metropolitan and nonmetropolitan medical schools. As indicated by the qualitative interview data, considering “the substantial human and material resources required for interview evaluations” and “the need for coordination and collaboration with university headquarters,” ensuring autonomy and independence in student selection at medical schools is crucial [16].
Second, the challenges faced by medical schools in conducting admissions interview evaluations and the strategies employed to overcome these challenges are reported. This study highlighted the difficulties faced by smaller medical schools in maintaining continuous pools of evaluators. Particularly in cases such as the MMI, which requires a significant number of evaluators, the shortage of personnel intensifies these challenges. Nevertheless, each university made efforts such as evaluator training, development programs, and fair evaluation through admissions committees and studies, in order to validate the effectiveness of the interview evaluations.
According to previous research, stakeholders in the interview process, such as interviewers, professor, and admissions committee members, universally consider fairness, validity, and comprehensiveness to be paramount aspects of the admission selection process. The importance of these elements has been emphasized, highlighting their critical role in ensuring the integrity and effectiveness of the overall admission evaluation process.
Above all, the interview assessment reflects the values and philosophies of each university, making it a tailored evaluation method for selecting the desired students. Therefore, continuous and systematic research, involving data collection and analysis on different interview methods, is essential for universities to determine the most effective interview approach [17].
Finally, based on the research findings, recommendations are made for student selection and interview operations in South Korean medical schools. Although there are not as many research papers as in other countries, South Korea is also examining the effectiveness of interview assessments—specifically, the relationship between medical school admission interview scores and academic achievements. However, unlike other countries that explore a variety of topics such as the biases perceived by medical school interview candidates and the operational costs of interview evaluations, South Korea has not diversified its research focus. It is necessary for South Korea to conduct more comprehensive research on interview evaluations to determine their effectiveness in South Korean medical schools and to identify necessary improvements.
This study has certain limitations that should be acknowledged. First, the findings are based on data collected within the context of the South Korean Medical School Admission System. Generalizing these results to other international settings or educational systems requires caution because of the unique characteristics of the South Korean context.
Second, the research primarily focused on operational aspects and experiences related to rolling out admissions in qualitative data analysis. Although this choice provided in-depth insights into this particular admissions type, it may limit the broader applicability of the findings to other admissions, such as regular admissions. The study did not specifically analyze the distinctions between rolling and regular admission processes. These limitations underscore the need for future research to explore the nuances of various admission types and consider the generalizability of the findings across diverse cultural and educational contexts.
In conclusion, the quantitative data presented in this study align with and reinforce the qualitative insights, providing a robust foundation for understanding the operational aspects and perceptions of medical school admission interview evaluations in South Korea. The integration of quantitative and qualitative data will enrich the discussion, by offering a comprehensive view of the challenges and strategies associated with the interview process in South Korean medical schools.
PDF Links
PubReader
ePub Link
XML Download
Full text via DOI
Download Citation 
Print 
