240 likes | 356 Views
The Effect of Interviewer on Rank List: An Imperfect Science Becomes More Imperfect. Daniel Vargo, MD Program Director, General Surgery Associate Professor, Dept. of Surgery University of Utah School of Medicine. Disclosures. None. Background. Applicant Selection: Surgery Job Description
E N D
The Effect of Interviewer on Rank List: An Imperfect Science Becomes More Imperfect Daniel Vargo, MD Program Director, General Surgery Associate Professor, Dept. of Surgery University of Utah School of Medicine
Disclosures • None
Background • Applicant Selection: Surgery • Job Description • Website • “Red Book” • Applications • 526 last year • Nebulous scoring system • Interviews
Background • Interviews • Interviewers not HR trained • Go on “gut feeling” • “I wanted to like/not like this candidate” • Spend interview trying to validate feelings or impressions • Trip up questions • Base opinion on “unusual” things…..
His socks didn’t match his pants Who wears a pants suit to an interview He had this weird look in his eyes
PGY-1 Summative Meeting • Interview Comments vs. Performance • No correlation • Lowest scored intern last two years highest performance
“Six Sigma” Evaluation • Took process apart • Biggest perceived variable • Interviews
Question • How variable are the interviews? • What effect does this variability have on process?
Methods • 5 years data • Interviewers and scores • “Easy Scorers” • “Hard Scorers” • Applicants • Strong • Average • Weak
Methods • Applicant group ranking • Compared with interview panel composition
Results • 30 Interviewers • 303 applicants • 909 interviews
Applicant Distribution Top 10 Appl. >40 Or NR
“HS” “ES”
Interview Scores:All Candidates ES= Easy Scorer HS= Hard Scorer
Scoring: Strong Applicants HS=Hard Scorer P= NS
Scoring: Average Applicants * p<0.05
Results • ≥ 1 HS • Lower interview scores • Lower composite scores • Lower position on rank list • Less likely to be discussed at rank meeting
Assumptions • Candidate pools are equally distributed • Interviewer “toughness” did not vary • Other variables in score calculation consistent
Conclusions • Interviewers do vary in type • Scores effect applicants • Another area of variability to be addressed in the interview process