Brandon L. Young, MA, PHR Manager of Testing and Measurement

1. The Real Power of Employee Tests and Assessments Brandon L. Young, MA, PHR Manager of Testing and Measurement

2. Agenda • The business need • Evidence of ROI • Achieving ROI • Measuring/Communicating ROI

3. What is a Test? • Anything used to select employees, including interviews, is a selection device and may be treated as a test. • Cognitive Ability • Job Knowledge or Skills • Personality • Physical Ability • Job Content - Simulations

4. Staffing Concerns • Hiring new employees is a big investment • Finding top candidates is time consuming • Companies are flooded with applicants • Who is qualified? Who is not qualified? • Bad hiring decisions and losing good applicants are problems • Not enough training dollars

5. The Stakes are too High Now “If the economy was better, we would have more of a margin for error. We don’t have that margin of error now.” —Pat Nangle,Actuarial Administration ManagerFarmers Insurance “Before, there was a little more tolerance to bringing people up to speed. Now, it’s ‘Who’s really on the bus?’” —Michael Scarpone,Scarpone & Associates

6. Employee Performance and Business Outcomes • Good hiring decisions have positive impact on business results • Poor hiring decisions • Negatively impact performance and productivity • Increase turnover and staffing costs • Inefficiently use managerial time • Lower customer satisfaction

7. Testing as a Solution • Many employers use skill tests and behavioral assessments to predict success in both the organization and the position. • Risk management for bad/negligent/careless hiring is reduced • Replaces “gut feel” with metrics and measures • Improves staffing efficiencies • Quantifies success characteristics

8. Best Measurement Methods Workers Competencies Resumes, Structured Interviews, Credentials, Skills Tests Knowledge Experience Skills Work Styles Personality Traits, Attitudes, Motivation Validated Assessments Supported by Current Legal and Professional Standards Cognitive Abilities Critical Thinking, Verbal, Numerical, Mechanical Holistic Assessment

9. Where to Start • Ascertain what positions are best served by assessment. • Look specifically at areas of high turnover or job dissatisfaction. • Determine the pivotal talent pools within the organization.

10. Best Practices • Desired end goals should be defined: • improved performance, increased productivity, improved process efficiency, reduced turnover, etc. • Financial investments should be considered prior to developing or purchasing selection tests. • “Is the potential information and use of the information worth the financial investment and what is the potential return on this investment?”

11. What is the ROI? • Benefits of better hiring decisions. • Benefits of avoiding “Regrettables” • Benefits of reduced turnover • Benefits of increased staffing efficiency

12. Case Study 1: CSR’s at Inbound Call Center • Retention rates were only 40% per year. • Internal benchmarking showed that Personality and Cognitive Assessment scores were statistically different between groups of retained and early terminating employees. • Within one year of using the new benchmark retention rates increased 100%.

13. Case Study 2: Home Health Professionals • Turnover at100% per year. • Predictive validation study, all applicants were tested but the test results were not available to the hiring manager. • After nine months, results analyzed against retention and performance data for those hired. • Assessments accurately predicted 83% of turnover with new employees.

14. Case Study 3: Major Staffing Company • Overall placement rate for a big client had been 33% of applicants presented as potential hires. • After the assessments became part of the staffing company’s internal hiring protocol, able to obtain the complete staffing responsibility for this client and experienced an astounding 100% placement rate for applicants presented to the client.

15. Case Study 4: Cable TV Sales Persons • Assessed cognitive and behavioral traits • In one year of using the assessments, reduced “90 day or less turnover” by 50% • In one year of using the assessments, reduced “training pay” by 30% to 50% in each division which allowed bottom line payroll savings to be re-allocated to improve training and development

16. Case Study 5: Sales Reps Visual Communications Company • Goal was to improve sales performance. • Cognitive and personality test administered to incumbents. • Incumbents separated into A, B, and C performers • Assessment identified 100% of C performers and 84% of A performers

17. Prediction • “How can we best predict Y?” - where Y is some meaningful and important aspect of workplace behavior. • Sales performance • Productivity • Client Retention • Training success rates • Turnover, Etc.

18. Job analysis Identify relevant job performance dimensions Identify KSAO’s necessary Identify/develop instrument(s) to measure KSAO’s Validation – Reliability, Validity (content, construct, criterion), Adverse Impact Use instrument(s) Continuously Evaluate

19. Job Analysis & Assessment • Identifying the criterion • Determining critical elements of a job for designing selection programs • Developing performance measures • Determining the focus within a particular assessment solution/predictor • Determining appropriate test/item weights • Conducting test transportability studies

20. Create a Test Plan

21. Reliability Types and Validity Approaches Test-Retest: Stability of test over time Reliability Parallel Forms: Equivalence of two test forms Internal Consistency: Consistency among test items Content: Test representative of domain Validity Construct: Test measures underlying principle Criterion-Related: Predictive: Test scores predict future criterion (Divergent, Convergent) Concurrent: Test scores predict current criterion (Divergent, Convergent)

22. Why Validity? • Critical to better decision making and legal defensibility • The validity of selection method is directly related to its practical value • Gains from increasing the predictive validity of hiring methods can increase bottom line results by millions of dollars • Using methods with low predictive validity can result in losing millions of dollars

23. No Predictive Accuracy Predictor Criterion

24. Moderate Predictive Accuracy Criterion Predictor

25. High Predictive Accuracy Predictor Criterion

26. Conduct a job analysis or collect extant job data. Benchmarking 1 • Analyze the content of the job analysis to identify core competencies. 2 • Identify measures, including tests, which can best measure the most critical competencies. 3

27. Identify three groups of incumbents based on identified competencies. • Top ,Average, and Poor or Struggling Performers. Benchmarking Cont. 4 Administer selected tests to all individuals in each of the three groups. 5 Examine three sets of data –determine the test scores most clearly differentiating top performers from the poor or struggling . 6

28. Reasons for Benchmarking • Provides management with information about the nature of a specific job and the competencies necessary to do that job successfully • Results of benchmarking should provide the necessary evidence of the job relevance and criterion validity

29. Maximizing Effectiveness • Decisions Regarding Use of Assessment Instruments • Combining different measurements • Establishing cut scores/passing standards • Sequencing of assessments in decision making • Administration issues

30. Evidence of Effectiveness • Utility Analysis • ΔU = N T SDy rxy λ(Q)/Q – C • Coefficient of determination • Percent decreases/increases in criterion • ROI

31. Communicating Potential ROI • Estimates of turnover are 30 Sales Representatives five times per year. • A conservative estimate for cost of turnover per Sales Representative is $30,000. • Using these estimates, the total cost of turnover is $4,500,000 per year.

32. Show me the Money • Total initial cost of benchmark and personality testing license - $10,000 • Ongoing costs include the annual personality testing licenses at $3,000. • Decreasing turnover by just one Sales Representative will save the company $20,000 in the first year and $27,000 in the following years.

33. Show me More Money • Decrease turnover by 10% - savings of $450,000 annually. • Decrease turnover by 25% - savings of $1.2M. • Decrease turnover by 50% - savings of $2.25M.

34. Communicating ROI Collect the Defined business Outcomes Analyze relationship between hires and business outcomes

35. Final Considerations • Psychometric Soundness • Validity • Reliability • Legal Defensibility • Job Analysis • Adverse Impact • Practicality • Economical • Usability

36. Practicality • Stakeholders = Candidates, HR Incumbents, Hiring Managers • Ease of use • Administration • Scoring • Interpretation • Costs - $ and time • Accuracy

37. MSEC Testing Services • Job analysis and documentation • Test identification or development • Online product suite • Skills Tests • Behavioral Assessments • Test review • Validation and benchmarking • Adverse impact analysis • ROI analysis

38. Additional Resources • MSEC • www.msec.org/msecservices/assessment_pes.asp • Locate Test Reviews • www.unl.edu/buros • www.testpublishers.org • Legal and Government Requirements • www.doleta.gov • www.eeoc.gov • Professional Standards • www.apa.org/science/standards.html • www.siop.org • www.online.onetcenter.org

39. Questions?

40. Contact Information Brandon L. Young, MA, PHR Manager of Testing and Measurement Mountain States Employers Council, Inc. 303.941.7760 or 1.800.884.1328 byoung@msec.org www.msec.org