(18) Validity Evidence for a Virtual Multiple Mini Interview

Objective : To examine the validity and reliability of virtual multiple mini-interviews (vMMIs) and explore differences in performance between vMMIs and in-person MMIs.

Methods: Data were collected for two years of in-person MMIs and two years of vMMIs. An exploratory factor analysis (principal components analysis) with varimax rotation and Kaiser rule (i.e. retaining factors with eigenvalue >1.0) was used to explore the construct validity of the vMMI data. Pearson correlation was used to examine correlations between vMMI stations and Cronbach alpha was used to determine the internal consistency of each station. Independent t-tests were used to examine differences between in-person MMI and vMMI scores. Cohen’s d was used to determine effect sizes.

Results: Four hundred and thirty-eight (42.69%) candidates completed an in-person MMI and 588 (57.31%) completed a vMMI. Factor analysis indicated that each vMMI station formed a single factor with loads ranging from 0.86 to 0.96. The vMMI stations accounted for most of the total variance, demonstrated weak to negligible intercorrelations, and high internal consistency. Significant differences between in-person and vMMI scores were found for the teamwork-giving, teamwork-receiving, and integrity stations. Medium effect sizes were found for teamwork-giving and teamwork-receiving and a small effect size was found for integrity.

Conclusions: Initial evidence suggests that the vMMI is a valid and reliable alternative to in-person MMIs. Additional research is needed to examine sources of potential differences in rating patterns between different interview modalities and formats.