Medical student selection interviews: insights into non-verbal observable communications: a cross-sectional study
Article information
Abstract
Purpose
Interviews play a crucial role in the medical school selection process, although little is known about interviewers’ non-verbal observable communications (NoVOC) during the interviews. This study investigates how interviewers perceive NoVOC exhibited by interviewees in two medical schools, one in Taiwan and the other in Australia. The study also explores potential cross-cultural differences in these perceptions.
Methods
A 26-item questionnaire was developed using a Delphi-like method to identify NoVOC. Interviewers from the University of New South Wales, Australia, and National Yang Ming Chiao Tung University, Taiwan (n=47 and N=78, respectively) rated these NoVOC between 2018 and 2021. Factor analyses identified and validated underlying factors. Measurement invariance across countries and genders was examined.
Results
A total of 125 interviewers completed the questionnaire, including 78 from Taiwan and 47 from Australia. Using exploratory factor analysis, 14 items yielded reliable three factors “charming,” “disengaged,” and “anxious” (Cronbach’s α=0.853, 0.714, and 0.628, respectively). The measurement invariance analysis indicated that the factor models were invariant across genders but significantly different between the two countries. Further analysis revealed inconsistencies in interpreting the “anxious” factor between Taiwan and Australia.
Conclusion
The three distinct factors revealed in this study provide valuable insights into the NoVOC that interviewers perceive and evaluate during the interview process. The findings highlight the importance of considering non-verbal communication in selecting medical students and emphasize the need for training and awareness among interviewers. Understanding the impact of non-verbal behaviors can improve selection processes to mitigate bias and enhance the fairness and reliability of medical student selection.
Introduction
Interviews are deemed an essential part of the medical school selection process as they assess a range of interpersonal skills (e.g., communication, empathy, and so forth) that cannot be gathered from other sources [1,2]. Medical school interviews can take various forms, including traditional one-on-one interviews, multiple miniinterviews, panel interviews, and group interviews [1].
Despite the popularity of medical selection interviews, their validity is debatable because interviewers form subjective judgments based on personal preferences, stereotypes, or biases [3]. Due to time constraints and limited information, interviewers often rely on heuristics or mental shortcuts to simplify decision-making [4]. These heuristics commonly involve initial impressions (anchoring bias or halo/horn effect) and discrepancies between verbal and non-verbal communication. Research on nonverbal cues suggests that dynamic vocalics, eye contact, head movement, and professional appearance significantly influence interview outcomes, whereas posture and smiling do not [5]. Notably, interviewers can only make accurate personality judgments through non-verbal cues when valid cues are available, observable, and align with their validity for specific personality traits. Despite the importance of this topic, no previous research has examined how interviewers’ heuristics influence evaluations in medical school selection interviews, which this study aims to address.
The main objective is to identify how interviewers perceive interviewees’ non-verbal behaviors in two medical schools in different cultural contexts, one in Taiwan and the other in Australia. The study will also explore whether interviewers’ impressions of non-verbal behaviors differ between the two countries.
Methods
1. Research design and ethics
This is a cross-sectional observational study, approved by the Human Research Advisory Panel (HC210695) of the University of New South Wales (UNSW), Sydney, Australia, and the Institutional Review Board (YM109109E) of the National Yang Ming Chiao Tung University (NYCU), Taipei, Taiwan.
2. Setting
Data were collected from interviewers participating in the medical school selection process in two medical schools, the UNSW and the NYCU. An online questionnaire link, participant information sheet, and consent form were sent to participants’ emails. The process was voluntary and no identifiable information was recorded. Data collected were digitally stored in the online platform.
3. Participants
All interviewers who participated in medical student selection in UNSW and NYCU between 2018 and 2021 were invited, and no exclusion criteria were applied. In total 47 and 78 interviewers from Australia and Taiwan, respectively, participated in the study.
4. Variables
Demographic variables included age group, gender, and occupation. The location was recorded as either Taiwan or Australia in accordance with the university. Factor extraction occurred during the analysis phase.
5. Data sources/measurement
A Delphi-like method was used with interviewers from UNSW and NYCU to identify non-verbal observable communications (NoVOC) during semi-structured interviews, focusing on the main question: “What non-verbal behaviors do you observe on top of the conversation during the interview? For example, what observations that you see, hear, and feel are considered.” Insights into nonverbal behaviors observed such as voice, gestures, facial expressions, and pre-interview observations were gathered. An inventory of these behaviors was created from 26 NoVOC descriptions identified, which were initially written in English and then translated into Traditional Chinese by a translator accredited by the National Accreditation Authority for Translators and Interpreters. A pilot study confirmed the clarity and appropriateness of the questionnaire items.
An online questionnaire was developed, incorporating the 26 NoVOC identified (Supplement 1). Participants were asked to indicate their general perception of each behavior during an interview by rating them on a scale of 1–10, ranging from “very negative” to “very positive impression.” Following the behavior ratings, age group, gender, and occupation were included as demographic questions.
6. Potential bias
Response bias might exist due to the nature of a questionnaire-based study. Given that the study includes two institutions from different countries, other biases such as gender, age, and cultural differences might be present [6,7], although these are investigated as part of the study objectives.
7. Study size
It is typically advised to recruit 10 participants per questionnaire item, with a suggested sample size of around 300 for both exploratory and confirmatory factor analyses (EFA and CFA), with a minimum of 150. However, it has been suggested that a minimum sample size of 100 is sufficient for delivering reliable results [8].
8. Statistical methods
EFA in IBM SPSS ver. 24.0 (IBM Corp., Armonk, USA) was performed using maximum likelihood with direct Oblimin rotation to identify underlying factors [9]. The scree plot determined the eigenvalue cut-off. Low-(<0.45) and cross-loaded items (>0.25 on other factors) were excluded to retain robust factor representations [10]. Cronbach’s α assessed the internal consistency and reliability of each factor. To confirm the construct validity of the identified model [11,12], CFA using IBM SPSS Amos ver. 24.0 (IBM Corp.) established an acceptable fit model, considering correlated errors due to high shared variance between items.
Reliability across countries and genders was examined through measurement invariance. Univariate analysis of variance (ANOVA) examined factor differences between countries, genders, and their interaction. To further investigate variations in interviewer impressions across countries, Cronbach’s α of each factor was examined separately for each country.
Results
1. Participants
The questionnaire was completed by 47 interviewers in Australia and 78 in Taiwan (Table 1). Demographic characteristics were significantly different between the participants across the two countries. Compared to participants from Australia, Taiwanese interviewers were younger, mostly male, and more likely to be medical doctors and academic staff.
Characteristics of the Study Population
2. Main results
1) Exploratory factor analysis
With the eigenvalue threshold set at 1.5, three factors were established through EFA. Eleven items with low factor loadings and another two with cross-loading (provides uncertain answers; sits with a straight back) were eliminated. One item (talkative in the interview) was removed due to less similar to the other items under the same factor, and 14 out of 26 items were retained. The factor loadings and groupings for each item are listed in Table 2. The first factor comprises items related to disengaged behaviors, the second includes behaviors perceived as charming, and the third represents behaviors indicating anxiety. The reliability (Cronbach’s α) for each factor was 0.853, 0.714, and 0.628, respectively, with the first two demonstrating good and acceptable values (α>0.8 and α>0.7).
Factor Loadings of the Items in Pattern Matrix
2) Confirmatory factor analysis
CFA was conducted using all 14 identified variables to construct a model with an acceptable fit and check the structure (Fig. 1). Two pairs of errors were correlated (items 2 and 25; items 21 and 25) due to high variance shared between the items. The model’s fitness is acceptable based on the chi-square (chi-square degrees of freedom ratio [CMIN/DF]=1.590). Additionally, the comparative fit index (CFI=0.992) and root mean square error of approximation (RMSEA=0.069) indicate the model’s fit is within an acceptable range. The “disengaged” factor was moderately correlated with the “anxious” factor (r=0.58), while the “charming” factor had lower correlations with both factors (r=0.11 and 0.05).
The Confirmatory Factor Analysis Using 14 Identified Nonverbal Observable Communications
3) Factor structure reliability across countries and genders
Multiple-group measurement invariance analysis identified invariance across genders (measurement weights: p=0.199), although the models were significantly different across countries (measurement weights: p=0.009). The results of univariate ANOVA are shown in Table 3, indicating no interactions between the country and gender variables. In separate analyses by country, a difference in reliability was found in the “anxious” factor, suggesting this factor is not consistent across the two countries (Table 4).
Univariate ANOVA of Three Identified Factors Examining Interaction in Countries and Genders
Reliability (Cronbach’s α) of Each Factor in Two Countries
Since“ anxious” appeared to possibly lead to the significant difference between countries, we once again conducted CFA and multiple-group invariance analysis without this factor (Supplement 2), whereby all indices were acceptable (CMIN/DF=1.561; CFI=0.885; RMSEA= 0.068). There was invariance between the two countries regarding measurement weights (p=0.105) and structural covariance (p=0.259), suggesting that a two-factor construct of “disengaged” and “charming” is valid across the two countries.
Discussion
Among interviewees’ NoVOC, we identified three factors related to their perception of non-verbal behaviors: “charming,” “disengaged,” and “Anxious.” The validity and reliability of the first two factors are confirmed across the two countries, whereas interviewers did not consistently interpret or perceive “anxious” between the two countries.
A key finding is interviewers’ similar perceptions of “charming” and “disengaged” non-verbal behaviors across the two countries. “Charming” was characterized by happiness, smiling, and concise responses, while “disengaged” exhibited mostly opposite behaviors. The “charming” NoVOC aligns with valid non-verbal cues used to assess extraversion, agreeableness, and intelligence traits [13]. Despite reflecting the opposite, “disengagement” might represent a relevant medical profession trait, whereas “charming” might indicate bias [14]. Given that doctors need to demonstrate effective patient engagement, identifying disengagement expressions is important for medical selection. Conversely, while being charming is not inherently negative, it has not been identified as an important medical profession trait, and therefore interviewers should be aware of charm’s impact on their judgment. Moreover, expressing charm has been associated with narcissism, along with “flashy and neat dress,” “self-assured body movements,” and “humorous verbal expression” [15]. Adding the impact of oral fluency, dominant facial expression, and gestures on popularity such as zero acquiescence suggests that charm is an important bias that might lead to interviewers favoring narcissist interviewees who prioritize their own needs above those of others [16].
Two main implications are derived from this research. First, interviewers need to be aware that a “charming” interviewee should be considered more carefully than a less charming person due to the association between personal charm and narcissism, which not be strongly appreciated in the medicine profession. Second, once these factors are formalized and placed in the public domain as selection criteria (engagement and charm) or explicitly not (anxiety), they might influence applicants’ interview preparation. While it is difficult to predict the outcome of such unintended consequences, it is not recommended to ignore these two important factors that affect the results of medical selection interviews. Accordingly, further research into this topic is warranted.
Although no country differences were identified regarding the impact of “charm” and “disengagement,” the non-verbal behaviors of being “anxious” (showing sadness, moving on the chair, and speaking with a stammer) differ across the two countries, potentially due to cultural differences. Interviewers in Taiwan rated “anxious” as unfavorable, while those in Australia perceived it as less negative. The high/low context difference might partly explain the inconsistent perception of “anxious” behaviors, as high context cultures such as China and Taiwan strongly rely on non-verbal cues and indirect communication, while low context cultures such as the United States and Australia prioritize direct and explicit communication [17-19]. Interviewers are widely used to selecting among culturally diverse students for medical and health profession education programs (e.g., Asian students comprise up to 50% of some medical school cohorts in Australia). Therefore, interviewers should be trained to understand cultural differences and their impact on their impressions of interviewees to enhance the validity and trustworthiness of the interview process.
Most previous studies have explored non-verbal behaviors in job application interviews, identifying a positive link between recruiter evaluations and applicant non-verbal behaviors, particularly eye contact, smiling, and speech modulation [5]. Such research has mostly been conducted through a direct comparison of the listed non-verbal behaviors in relation to selection outcomes. By contrast, our study has followed a scale development methodology, including item development through a qualitative Delphi-like process, EFA to identify factors, CFA for model fit, multiple-group invariance analysis across populations, and reliability tests for internal consistency. This careful approach enhances confidence that the factors identified truly represent interviewers’ biases.
The main limitation is the sample size, as typically a larger sample size is required for such a study. The sample size in this study was smaller than expected (N=125), although the results of both EFA and CFA yielded an acceptable model fit. Replication of this study with a much larger sample size might be necessary to ensure that the findings are not incidental but rather representative. Our study identified a difference between the two countries in the factor of “anxious,” while “disengaged” and “charming” remained relatively stable in the invariance analysis. The latter two factors might have broader generalizability, although further studies are warranted for generalizability.
Future research can further explore the influence of non-verbal factors on selection outcomes and investigate strategies for effectively training interviewers to interpret and evaluate these cues, taking cultural differences into account. Additionally, if the use of the questionnaire becomes publicly known, it might affect how applicants prepare for interviews, necessitating further investigation.
In conclusion, the three distinct factors revealed in this study—“charming,” “disengaged,” and “anxious”—provide valuable insights into the NoVOC that interviewers perceive and evaluate during the interview process. The findings highlight the importance of considering nonverbal communication in selecting medical students and emphasize the need for training and awareness among interviewers. By understanding cultural differences and their impact on non-verbal behaviors, selection processes can be improved to mitigate bias and enhance the fairness and reliability of medical student selection.
Supplementary materials
Supplementary files are available from https://doi.org/10.3946/kjme.2025.332
Factor Loadings of All Items in Pattern Matrix and Justification of Deletion.
The Confirmatory Factor Analysis Using the Two factors, “Charmin” and “Disengaged.”
Notes
Acknowledgements
The authors are grateful to all the interviewers who anonymously participated in the present study.
Funding
This research was funded by the UCAT ANZ Consortium, Australia. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Conflicts of interest
No potential conflict of interest relevant to this article was reported.
Author contributions
Conceptualization: PHH, BS, YFC. Data curation: PHH, BS, YFC, KCF. Methodology/formal analysis/validation: PHH, BS, YFC. Project administration: BS, YFC. Writing–original draft: PHH, YFC, KCF. Writing–review & editing: PHH, YFC, AW, BS, KCF. Final approval of the manuscript: all authors.