ROI in HRM: Validity of Selection Procedures

1. ROI in HRM: Validity of Selection Procedures Nicole Blake Edgar Nolasco Deena Desai

2. Employee Selection: Will Intelligence and Conscientiousness Do the Job? Orlando Behling

3. Key Terms • General Intelligence “g” • Big Five • Meta-Analysis – combining research results from existing studies that meet a specific criteria for to generate a general conclusion.

4. Summary • Measuring general intelligence “g” in addition to the Big Five provides a stronger predictor of job performance in past jobs. • Group Hypothesis: What have we learned since Behling?

5. Article A Meta-Analytic Investigation of Conscientiousness in the Prediction of Job Performance: Examining the Intercorrelations and the Incremental Validity of Narrow Traits By: Nicole M. Dudley, Karin A. Orvis, Justin E. Lebiecki and Jose M. Cotina George Mason University Nicole Blake

6. Key Terms • Global Conscientiousness: A broad characteristic of being conscientious (careful, thorough). • Four Narrow traits of conscientiousness: • Achievement • Order • Dependability • Cautiousness Nicole Blake

7. Introduction: • The Big Five • Conscientiousness: the most important & the most valid personality predictor of job performance. • Multiple meta-analyses from previous research examined and used in the current meta analyses. • Goal of this study Nicole Blake

8. Hypothesis • 1) The narrow traits of conscientiousness are a better predictor than global conscientiousness alone. • 2) Narrow traits can be a useful predictor for specific occupations but not all occupations. Nicole Blake

9. Method • Validity coefficients and intercorrelations among the 5 predictors were obtained from previous studies. • Criteria for inclusion • 2 moderators: type of performance criterion & occupational type. • Hierarchical regression analysis Nicole Blake

10. Results 1: Incremental Validity for Job Performance Criteria Nicole Blake

11. Results 2: Overall Job Performance by Occupational Type Nicole Blake

12. Resume Screening & Personality Testing Recruiters Inferences of Applicant Personality Based on Resume Screening: Do Paper People have a Personality? By: Michael S. Cole, Hubert S. Feild, William F. Giles, Stanley G. Harris. • Excludes potential candidates. • Considerable error present in recruiters inferences. Nicole Blake

13. Take Home Message • Focusing on narrow traits is a better predictor of job performance than focusing on broad traits. • Managers should consider narrow traits when the goal is to predict contextual performance or to predict overall performance for specific occupations • Saves money in the long run by hiring the best fit applicants. Nicole Blake

14. Article Situational Judgment Tests, ResponseInstructions, and Validity: A Meta-Analysis By: Michael A. McDaniel, Nathan S. Hartman, Deborah L. Whetzel and W. Lee Grubb III Edgar Nolasco

15. Key Terms • SJT- Situational Judgment Test • Behavioral tendency • Knowledge • Cognitive ability or g – It is a set of abilities, skills or processes that are part of nearly every human action. • Big Five– Extraversion, Emotional Stability, Agreeableness, Conscientiousness and Openness. Edgar Nolasco

16. Introduction/Study • Two things being tested through this study • Using a Meta-Analysis • The psychometric artifact method was used in the study • Data set from the McDaniel (2001) study • Previous research shows that SJTs predict performance because they measure job knowledge Edgar Nolasco

17. Example of an SJT You have just attended a training course delivered by your manager. While talking to the others privately after the course, several of them were critical of how your manager delivered the course. Your manager is unaware of this feedback and you felt the course delivery was adequate. Rate the effectiveness of each of the actions below on the following scale: 1 = Very Ineffective, 2 = Ineffective, 3 = Effective, 4 = Very Effective. a. Tell your manager that you personally think the course went well and don’t mention the feedback from the other delegates. b. Tell the other delegates that they should communicate their feedback to your manager if they feel strongly about it.c. Tell your manager about the feedback from the other delegates, without mentioning them by name. d. Don’t mention any feedback about the course to your manager. Edgar Nolasco

18. Method • Study uses: • Data from McDaniel (2001) • The psychometric artifact distribution meta-analysis method (Hunter and Schmidt, 1990, 2004) • Criteria: • Employees or applicants only • Paper-Pencil format • Priority to supervisor ratings over job performance • Hierarchical level, # of employees supervised, years of management experience, and salary omitted Edgar Nolasco

19. Results : Criterion-Related and Incremental validity Edgar Nolasco

20. Take Home Message • When looking for the candidate both behavior and knowledge based SJTs should be asked. • SJTs that use behavioral instruction are easier to manipulate because respondents know what to say. • SJTs are unique: either serve assessment of typical or maximal performance • SJTs offer job relevance, ease of use, flexible, are validated and are fair Edgar Nolasco

21. Further Evidence for the Validity of Assessment Center Dimensions: A Meta-Analysis of the incremental Criterion-Related Validity of Dimension Ratings John P. Meriac, Brian J. Hoffman David J. Woehr and Matthew S. Fleisher Deena Desai

22. Key Terms Assessment Centers (AC) – objective method of measurement of skills, competencies and characteristics for a specific job Dimensions – Characteristics beyond the Big Five and cognitive ability Hierarchical Regression – adds factors at each stage Forward selection – testing individual variables Deena Desai

23. Introduction • Assessment Centers (AC’s) • Weak construct validity • Independent influence on job performance Deena Desai

24. Hypothesis How much if any, incremental variance in job performance do AC dimension ratings account for over and above that accounted for by cognitive ability and personality? Deena Desai

25. Method • Perform a Meta-Analysis • Retrieve studies that meet criteria • Extract AC dimensions • Input in one of the seven dimension categories identified by Arthur et. al. • Forward selection Deena Desai

26. Sample Breakdown Deena Desai

27. Sample Breakdown Deena Desai

28. Results DeenaDesai

29. Results • AC dimensions have an incremental variance in job performance above that of personality and cognitive ability • The individual characteristics in addition to personality and cognitive ability have a significant relationship with job performance • Cognitive ability + the big five + organization and planning = most significant impact on job performance Deena Desai

30. Take Home Message AC’s are costly therefore should not be used for all positions AC’s must utilize up to date measures to be effective Further research is required to obtain alternative methods that effectively measure specific characteristics Deena Desai

31. Group Take Home • The research demonstrates that to date: • Specific measurements of narrow traits beyond personality and cognitive ability are the strongest predictors of job performance. • Assessing general characteristics for a variety of jobs increases the risk of litigation.