Performance Prediction: Truths and Falsehoods

1. Performance Prediction: Truths and Falsehoods Dr. Jordan B Peterson Professor of Psychology University of Toronto

2. What do psychologists do? • Pursue scientific truth • Pursue careers

3. How to pursue scientific truth • MEASUREMENT • If the phenomenon cannot be measured • It does not exist

4. How to pursue scientific truth, continued • CONSTRUCT VALIDATION • Multimethod, multitrait • Analogous to sensory analysis • Five senses, not one • Plus intrasubjective measurement • Plus technological extension of sensory analysis

5. Use statistics destructively • Aim at demolishing your effect

6. How to pursue a career • Invent a scale (or a construct) • Or rename a scale that already exists • Refuse to validate it • No convergence • No divergence • No criterion • No multimethod, multitrait analysis • Correlate it with some phenomenon

7. How to pursue a career, continued • Publish the paper • Establish a small primate dominance hierarchy • Based on the scale • Climb the hierarchy • Use statistics selectively • Very selectively • Enjoy the fruits of your success! • But seriously distort the knowledge base • And spend your time chasing red herrings

8. NATURE | NEWS FEATURE • Replication studies: Bad copy • In the wake of high-profile controversies, psychologists are facing up to problems with replication. • Ed Yong • 16 May 2012

9. Things that might not exist • Self-esteem • (neuroticism – extraversion) • 25000 published papers • Whole political and educational agendas • Working memory and EF • IQ? • Emotional intelligence • Agreeableness

10. Things that might not exist • Grit • MacArthur Genius Grant (Angela Duckworth) • Martin Seligman (positive psychology) • Conscientiousness • Optimism/Pessimism • Extraversion/neuroticism • Promotion/prevention • Extraversion/conscientiousness

11. Things that might not exist • Empathy • Measures do not correlate well, but distill to agreeableness • Psychopathy • Agreeableness, negative, minus conscientiousness • Predatory parasite

12. Things that might not exist • Positive illusions • Extraversion, by Taylor’s own analysis • Practical intelligence • Sternberg is a crook • Multiple intelligences • Gardner is a crook

13. Things that might not exist • Any questionnaire measure independent of the Big Five • Any measure of cognitive function independent of IQ • Any behavioral measure of a trait, independent of IQ and the Big Five

14. Things that might exist

15. Individual Differences • Contextual and individual factors • Contextual • Cultural factors • Peer networks • Social support • Individual • Personality • Cognitive ability

16. Personality

17. TraitsMeasurementItemsEmployment VariablesEconomic Value

18. Traits

19. Industriousness Orderliness Conscientiousness Volatility (r) Withdrawal (r) STABILITY Emotional Stability Politeness Compassion Agreeableness

20. Creativity Intellect Openness PLASTICITY Assertiveness Enthusiasm Extraversion

21. Personality • Extraversion • Happiness, optimism, enthusiasm, gregariousness, assertiveness • Neuroticism • .2-.3 with major life outcomes • Particularly anxiety and depression, since that’s what it measures • Agreeableness • Compliance, empathy, maternality, kinship • criminality • Conscientiousness • .4 with major life outcomes • Openness • Creativity, intelligence • Disagreeable/neurotic • Personality disorders • Gender differences • 75% classification accuracy

22. Emotion & Motivation • Extraversion • Incentive reward (hope, curiosity, play, enthusiasm)/dopamine • Harvest attention/approach and exploit social situations • Neuroticism • Pain, frustration, disappointment, fear, anxiety/GABA, Serotonin • Avoid threat, uncertainty and punishment • Agreeableness • Empathy, sympathy, compliance, CARE (Oxytocin) • Form intimate relationships and share • Vs • Pursue individual agenda/defend territory • People vs things • Conscientiousness • Orderliness: DISGUST • Industriousness: GUILT, SHAME • Maintain order • Stay uncontaminated and sparkly clean • Implement goals • Openness • Manipulate abstractions prior to implementation • General Cognitive Ability • Awe, curiosity • Non-social incentive reward/dopamine

23. Models are grounded in motivation Capitalize on social groups Maintain order

25. Measurement

26. Items

27. Measurement • Trait Measures • Big Five Aspect Scale • Unfakeable Big Five • General cognitive ability • Fluid • Crystallized • Dorsolateral prefrontal • Creativity • Creative Achievement Questionnaire • Divergent thinking and fluency

28. (BFAS) Conscientiousness • Industriousness • Waste my time (r) • Always know what I am doing • Orderliness • Am not bothered by disorder (r) • Follow a schedule

29. (BFAS) Emotional Stability • Volatility • Get angry easily • Keep my emotions under control • Withdrawal • Seldom feel blue • Am filled with doubts about things

30. (BFAS) Agreeableness • Politeness • Respect authority • Believe that I am better than others (r) • Compassion • Am not interested in other people’s problems (r) • Take no time for others (r)

31. (BFAS) Openness • Creativity • Believe in the importance of art • Love to reflect on things • Intellect (but see general cognitive ability) • Am quick to understand things • Can handle a lot of information

32. (BFAS) Extraversion • Assertiveness • Take charge • Lack the talent for influencing people (r) • Enthusiasm • Make friends easily • Am hard to get to know (r)

33. Unfakeable Big Five (+)

34. Unfakeable Big Five (-)

35. Cognitive Ability • The complexity of the world • Motivated ends must be met • Modeling of possibility prior to implementation • Models generate solutions to three problems: • Where are you? • Where are you going? • How are you going to get there? • Allows testing and failure • Popper: “Our hypotheses die in our stead”

36. Conceptions of cognitive ability • INTELLIGENCES • Practical vs analytical • Social, emotional, moral • Multiple: linguistic, musical, logical-mathematical, spatial, body-kinesthetic, intrapersonal, interpersonal • INTELLIGENCE • Highest strata has the most explanatory power • “g” measures “success” across broad domains • r ~ .50

38. Relative magnitude r= .50 90-97%ile r= .35-.50 75-90%ile r= .15-.35 25-57%ile r< .15 0-25%ile Hemphill, J.F. (2003). Interpreting the magnitudes of correlation coefficients. American Psychologist, 58, 78-8. Binomial Predictor r = .30 From 50/50 To 65/30 Predictor r = .50 From 50/50 To 75/25 Effect Size

39. General Cognitive Ability

40. Creativity Creative Achievement Questionnaire • Visual Arts (painting, sculpture) • 0. I have no training or talent in this area. • 1. I have taken lessons in this area. • 2. People comment on my talent in this area. • 3. I won one or more prizes at juried art shows. • 4. I showed my work in a gallery. • 5. I sold a piece of my work. • 6. My work was critiqued in local publications. • 7. My work was critiqued in national publications. • Inventions • 0. I do not have recognized talent in this area. • 1. I find novel uses for household objects. • 2. I sketched out an invention and worked on its design flaws. • 3. I created original software for a computer. • 4. I built a prototype of one of my designed inventions. • 5. I sold an inventions to people I know. • 6. I received a patent for one of my inventions. • 7. I sold an invention to a manufacturing firm.

41. Employment Variables

42. Hi/Hi Entrepreneurial Hi/Lo Innovation Lo/Hi Managerial/Administrative Lo/Lo Rote Tasks Complexity

43. Who suits which position? • Cognitive ability • High levels of fluid and executive intelligence • Decision making and planning • Research • High levels of retrieval • Creativity and ideational production • Leadership and public speaking • High levels of crystallized intelligence • Information dispensing and storage • Writing and lecturing

44. Who suits which position? • Specific Performance • Openness (r = .30 - .40) • Associated with high levels of creativity/entrepreneurial ability • Extraversion (r = .20) • Managers • Sales People • Leadership • Agreeableness • Lower levels associated with creativity/managerial excellence • Higher levels for public relations, customer service (?)

45. Openness • Complexity and Intelligence • Innovation and Creativity

46. Conscientiousness • Duty and Industriousness • Disgust and Orderliness

47. Employment Variables II • Emotional Stability • High vs Low Stress • Agreeableness • People vs Things • Extraversion • Solitary vs Gregarious

48. Economic Value

49. 95-86 percentile IQ 130 – 116 85-73 percentile IQ 115-110 Attorney, Research Analyst Editor, Advertising Manager Chemist, Engineer, Executive Manager, Trainee, Systems Analyst, Auditor Copywriter, Accountant Manager/Supervisor Sales Manager Sales, Programmer Analyst, Teacher, Adjuster General Manager Purchasing Agent Registered Nurse Sales Account Executive Complexity and cognitive ability

50. 70-60 percentile IQ 108-103 55-50 percentile IQ 102-100 Administrative Assistant Store Manager, Bookkeeper Credit Clerk. Drafter, Designer Lab Tester/Tech, Assistant Manager General Sales, Telephone Sales Secretary, Accounting Clerk, Medical Debt Collection Computer Operator Customer Service Rep Technician, Automotive Salesman Clerk, Typist Dispatcher, General Office Police Patrol Officer Receptionist, Cashier General Clerical Inside Sales Clerk, Meter Reader Printer, Teller, Data Entry Electrical Helper Complexity and cognitive ability