Measuring Quality-of-Hire

1. Measuring Quality-of-Hire Claude G. Balthazard, Ph.D. Director of Research HR.com cbalthazard@hr.com

2. Poll #1 Does your organization currently measure quality-of-hire? • Yes • No • I don’t know

3. Measuring Quality-of-Hire AGENDA • Context • Basics: How to measure quality-of-hire • What to do with quality-of-hire information once you have collected it • Improving quality-of-hire without measuring quality-of-hire? • Talentship and utility analysis

4. “let me be clear, the single most important thing you can do to improve ANY staffing function is to measure and reward the quality/performance of the people you hire. Everything else pales by comparison in changing the way you recruit!” Dr. John Sullivan Electronic Recruiting Exchange October 22, 1999

5. QUALITY OF HIRE SPEED OF HIRE COST OF HIRE Big 3 Staffing Metrics “Ultimately, quality of hire has the potential for the greatest impact on the organization” Source: Lermusiaux, Y., & Snell, A. (2004) Quality of Hire: The Next Edge in Corporate Performance. iLogos Research. p. 12

6. What is Quality? • Quality-of-hire refers to the productivity potential of a employee • Quality-of-hire = talent • Whether this potential is ever actualised is another matter however

7. Time-to-Contribution Productivity Level Employee A Minimum Standard of Output Employee B Time Time to Fill

8. Paradigm Shift • Efficiency and effectiveness • ‘Upsetting the apple cart’ • Quality as ‘meeting specs’ to something ‘open ended’ • Quality-of-hire is the metric we say we ‘should’ measure but still don’t (mostly)

9. “HR measures largely focus on the traditional paradigm of delivering HR services quickly, cheaply, and in ways that satisfy clients.” Boudreau, J. W., & Ramstad, P. M. (2003) Source: Strategic HRM Measurement in he 21st Century: From Justifying HR to Talent Leadership In HRM in the 21st Century (pp. 79-90), M. Goldsmith, R. P. Gandossy, & M. S. Effron (Eds.), New York: Wiley.

10. Quality-of-Hire Top Five Issues • No consistent way to define quality-of-hire • Time lag • Subjectivity • Other influences obscure the contribution of talent • Standardization

11. Measuring Quality-of-Hire 1. Output and performance of the hire 2. Subjective assessment by managers and others 3. Percent of qualifications met 4. Compensation and promotional rewards given the hire 5. Retention rates of top performers Source: Dr. John Sullivan The Only Recruiting Measure that Matters--Measure the Quality of your Hires Electronic Recruiting Exchange October 22, 1999

12. "We recommend completing the New Hire Quality Evaluation three to six months after hire. It takes at least three months before a new hire performance can be accurately assessed. After 6 months, work place influences become a dominant factor in new employee performance" Nick Burkholder Frameworks & Metrics: The Basics January 2004 www.staffing.org

13. Get On With It • The advice of most experts is to “get on with it” • Get over the fact that quality-of-hire metrics may not be as tight as Cost-per-Hire and Time-to-Fill metrics • Use multiple measures to ‘triangulate’ quality-of-hire

14. How Many Organizations are Actually Measuring Quality-of-Hire? • In 2005, the Aberdeen Group found that 52% of surveyed organisations indicated that quality-of-hire is a key indicator of hiring success • However, the Aberdeen Group “sees this response as an intent, rather than an indication that the companies are actually systematically measuring the quality of the hires.” Source: The Aberdeen Group The Enterprise Talent Management Benchmark Report: Hiring Right, Hiring Smart. March 2005

15. HR.com’s Surveyon Talent Management Metrics 0 0 0 10 10 10 20 20 20 30 30 30 40 40 40 50 50 50 60 60 60 70 70 70 80 80 80 90 90 90 100 100 100 Whether an offer was made Whether an offer was made Whether the offer was accepted or rejected by the candidate Whether the offer was accepted or rejected by the candidate The starting salary The starting salary Candidate source Candidate source Number of Applications per position Number of Applications per position Some aspects of applicants’ background Some aspects of applicants’ background Characteristics of position being filled Characteristics of position being filled Geographical location of position Geographical location of position Time to fill position Time to fill position Some measure of applicant quality Some measure of applicant quality If the offer was rejected, the reason why it was If the offer was rejected, the reason why it was Costs associated with the staffing process Costs associated with the staffing process EEO status EEO status Hiring manager satisfaction with the staffing process Hiring manager satisfaction with the staffing process Post-hire measure of performance Hiring manager satisfaction with the staffing process Hiring manager satisfaction with the staffing process Candidate satisfaction with the staffing process Candidate satisfaction with the staffing process Quality-of-Hire Metric 0 0 0 10 10 10 20 20 20 30 30 30 40 40 40 50 50 50 60 60 60 70 70 70 80 80 80 90 90 90 100 100 100 Manual System Manual System Online System Online System Percent Percent

16. Current Status • Quality-of-hire is the metric that we say we should measure but don’t • It depends on what is meant by ‘measuring quality-of-hire’ • Efficiency metrics still rule

17. Poll #2 On a scale of 1 to 5, where 1 refers to no measurement of quality-of-hire, and 5 refers to quality-of-hire being the metric around which all of our staffing activities are organized. How would you rate current practice in your organization? • 1 • 2 • 3 • 4 • 5

18. What to do with quality-of-hire information once you have collected it

19. “The ultimate best practice for staffing to improve the quality of hire is to use quality of hire data to optimize the staffing process. Such an optimization involves a rigorous ‘loop back’ of quality of hire measurements to changes made in the staffing process.” Yves Lermusiaux & Alice Snell Quality of Hire: The Next Edge in Corporate Performance iLogos Research 2004

20. Looping Back • Impressions are notoriously unreliable as evidence that this or that factor are drivers of quality-of-hire • We tend to notice instances that confirm our hypotheses and discount instances that disconfirm our hypotheses

21. Looping Back Candidate Characteristics Variables pertaining to the staffing process Quality-of-Hire Variables pertaining to channel Variables pertaining to the position being filled Variables pertaining to the onboarding process

22. Four Biggest ChallengesIn Using Quality-of-Hire Data to Improve the Staffing Process • Disentangling the various potential contributors to quality-of-hire • The ‘accuracy metric’ • Making allowances for noise in the system • Stuck inside the box

23. Disentangling Drivers • Quality-of-hire is at the end of a process that has many steps • Quality-of-hire metrics can give an indication that “something just isn’t right” but a diagnosis will usually require some digging

24. A Supply Chain Approach Post-Hire Quality-of-Hire Assessment Applicant Pool Offer Acceptance Pre-Screening Assessment Short List • Each step should act to ‘distill’ quality • Quality is something that exists throughout the chain • At each step, lower-quality candidates should be discarded and higher-quality candidates retained

25. The Accuracy Metric A B Successful C D Not Successful Not Selected Selected

26. A B Successful C D Not-successful Not Selected Selected

27. 21 7 Successful 9 3 Not-successful Not Selected Selected

28. The true appreciation of the impact of a given factor on quality-of-hire would require that we look those ‘other two cells,’ which, of course we don’t have

29. Noise in the System • Many fail to appreciate how noisy relations are in reality • The slice-and-dice approach can lead to small numbers • The Validity Exchange story and meta-analysis

30. Stuck Inside the Box • Of course, no analysis can tell us about possibilities that were not present in our dataset • No analysis can tell you about the source you didn’t use, the assessment instrument you didn’t use, etc.

31. Conclusion • In all but exceptional circumstances, we are unlikely to be able to pull together the kind and quality of data that would allow us to ‘loop back’ quality-of-hire data so as to optimise the staffing process • Using poor data as a basis for decisions can be worse than not using data at all

32. Improving quality-of-hire without measuring quality-of-hire?

33. In the context of optimizing one’s staffing processes, using someone else’s higher quality data is better than using one’s own poorer quality data.

34. Meta-Analysis • Meta-analysis is a methodology used to combine findings from multiple studies • The idea is to explain the variability in results across studies • A common finding is that much of the variability across studies can be accounted by chance variation inherent in small samples

35. Talentship and Utility Analysis • Utility analysis was, and is, a methodology that promised to calculate in dollars the impact of selection decisions • Quality-of-hire is embedded in utility analysis • At the core of utility analysis functions is the strength of the relation between quality-of-candidate measures used to make selection decisions and quality-of-hire measures

36. Selection Utility Functions The classic Brogden-Cronbach-Gleser model Where: DU Total utility gain (total monetary value gain) Ns Number of persons selected T Average expected tenure SDy Standard deviation of job performance measured in monetary value rxy Validity coefficient for selection procedure Zx Mean score of those applicants selected using the selection procedure C Cost of personnel selection

37. “Talentship is the decision science that improves organizational performance by enhancing decisions that impact or depend on human capital.” Boudreau, J. W., & Ramstad, P. M. (2003) Source: Strategic industrial and organizational psychology and the role of utility analysis models. In Borman, W. C., Ilgen, D. R., & Klimoski, R. J. (Eds.) Handbook of psychology: Industrial and organizational psychology, Vol. 12 (pp. 193-221)

38. Summing Up • Quality-of-hire should be the dominant concern for all staffing processes. The ROI of staffing is driven mostly through quality-of-hire • Measuring quality-of-hire can and should be done, but it is only the first step • The quality of the data that we can get our hands on is often not of the quality that will allow us to optimize staffing processes

39. Measuring Quality-of-Hire Q & A Claude G. Balthazard, Ph.D. Director of Research HR.com cbalthazard@hr.com