Work in the 21 st Century Chapter 2

1. Work in the 21st CenturyChapter 2 Methods and Statistics in I-O Psychology

2. Module 2.1: Science • What is science? • Approach that involves the understanding, prediction, and control of some phenomenon of interest • Science has common methods • Science is a logical approach to investigation • Based on a theory, hypothesis, or basic interest • Science depends on data • Gathered in a laboratory or the field

3. Common Methods (cont'd) • Research must be communicable, open, & public • Research published in journals, reports, or books 1) Methods of data collection described 2) Data reported 3) Analyses displayed for examination 4) Conclusions presented

4. Common Methods (cont'd) • Scientists set out to disprove theories or hypotheses • Goal: Eliminate all plausible explanations except one • Scientists are objective • Expectation that researchers will be objective & not influenced by biases or prejudices

5. Role of Science in Society • Expert witnesses in a lawsuit • Permitted to voice opinions about organizational practices • Often a role assumed by I-O psychologists

6. Module 2.1 (cont'd) • Why do I-O psychologists engage in research? • Better equip HR professionals in making decisions in organizations • Provide an aspect of predictability to HR decisions

7. Module 2.2: Common Research Designs in I-O Psychology Table 2.1 Common Research Designs in I-O Psychology

8. Research Designs (cont'd) • Experimental • Random assignment of participants to conditions • Conducted in a laboratory or the workplace • Non-experimental • Does not include manipulation or assignment to different conditions • 2 common designs: • Observational design: Observes and records behavior • Survey/Questionnaire design (most common) • Quasi-experimental • Non-random assignment of participants to conditions

9. Methods of Data Collection • Quantitative methods • Rely on tests, rating scales, questionnaires, & physiological measures • Yield results in terms of numbers C. Borland/PhotoLink/Getty Images

10. Methods of Data Collection: Qualitative & Quantitative Research • Qualitative methods • Include procedures like observation, interview, case study, & analysis of written documents • Generally produce flow diagrams & narrative descriptions of events/processes • Quantitative methods • Rely on tests, rating scales, and physiological measures • Yield numerical results

11. Quantitative & Qualitative Research (cont’d) • Not mutually exclusive • Triangulation • Examining converging information from different sources (qualitative and quantitative research).

12. Generalizability in Research Generalizability: • Application of results from one study or sample to other participants or situations • The more areas a study includes, the greater its generalizability • Every time a compromise is made, the generalizability of results is reduced

13. Sampling Domains for I-O Research Figure 2.1: Sampling Domains for I-O Research

14. Control in Research • Experimental control • Eliminates influences that could make results less reliable or harder to interpret • Statistical control • Statistical techniques used to control for the influence of certain variables

15. Ethical Behavior inI-O Psychology • Ethical standards of the APA • SIOP book of 61 cases (Lowman, 1998) • Cases illustrate ethical issues that are likely to arise in I-O psychology • Joel Lefkowitz (2003) published a recent book on values and ethics in I-O psychology

16. Table 2.2 Potential Roles Available to the I-O Psychologist and Other HR Managers with Respect to Ethical Problems

17. Module 3: Data Analysis • Descriptive statistics • Summarize, organize, describe sample of data Frequency Distribution: • Horizontal axis = Scores running low to high • Vertical axis = Indicates frequency of occurrence

18. Describing a Score Distribution • Measures of central tendency • Mean • Mode • Median Ryan McVay/Getty Images

19. Describing Score Distribution (cont'd) • Variability • Standard deviation • Lopsidedness or skew • Mean is affected by high or low scores, median is not • Mean pulls in direction of skew Ryan McVay/Getty Images

20. Descriptive Statistics:Two Score Distributions (N = 30) Figure 2.2 Two Score Distribution (N = 30)

21. Two Score Distributions (N = 10) Figure 2.3. Two Score Distributions (N = 10)

22. Inferential Statistics • Aid in testing hypotheses & making inferences from sample data to a larger sample/population • Include t-test, F-test, chi-square test

23. Statistical Significance • Defined in terms of a probability statement • Threshold for significance is often set at .05 or lower • Significance refers only to confidence that result is NOT due to chance, not strength of an association or importance of results.

24. Statistical Power • Likelihood of finding statistically significant difference when true difference exists • The smaller the sample size, the lower the power to detect a true or real difference

25. Concept of Correlation Positive Linear Correlation Figure 2.4 Correlation between Test Scores and Training Grades

26. Concept of Correlation (cont'd) • Scatterplot • Displays correlational relationship between 2 variables • Regression • Straight line that best “fits” the scatterplot and describes the relationship between the variables in the graph

27. Correlation Coefficient • Statistic or measure of association • Reflects magnitude (numerical value) & direction (+ or –) of relationship between 2 variables • Ranges from 0.00 and 1.00

28. Correlation Coefficient • Positive correlation → As one variable increases, other variable also increases & vice versa • Negative correlation → As one variable increases, other variable decreases & vice versa

29. Scatterplots of Various Degrees of Correlation Figure 2.6. Scatterplots of Various Degrees of Correlation

30. Curvilinear Relationship • If correlation coefficient is .00, one cannot conclude that there is no association between variables • A curvilinear relationship might better describe the association

31. Curvilinear Correlation Figure 2.7 An Example of a Curvilinear Relationship

32. Multiple Correlation • Multiple correlation coefficient • Overall linear association between several variables & a single outcome variable

33. Meta-Analysis • Statistical method for combining results from many studies to draw a general conclusion • Statistical artifacts • Characteristics of a particular study that distort the results • Sample size is typically the most influential statistical artifact

34. Module 4: Interpretation • Reliability • Consistency or stability of a measure • Test-retest reliability • Calculated by correlating measurements taken at Time 1 with measurements taken at Time 2

35. High and LowTest-Retest Reliability Figure 2.8 Examples of High and Low Test-Retest Reliability: Score Distributions of Individuals Tested on Two Different Occasions

36. Reliability (cont'd) • Equivalent forms reliability • Calculated by correlating measurements from a sample of individuals who complete 2 different forms of same test • Internal consistency • Assesses how consistently items of a test measure a single construct

37. Reliability (cont'd) • Inter-rater reliability • Can calculate various statistical indices to show level of agreement among raters • Values in the range of .70 to .80 represent reasonable reliability • Generalizability theory • Simultaneously considers all types of error in reliability estimates

38. Validity • Whether measurements taken accurately & completely represent what is to be measured • Predictor • Test chosen or developed to assess identified abilities or other characteristics (KSAOs) • Criterion • Outcome variable describing important performance domain

39. Figure 2.9: Validation Process from Conceptual and Operational Levels Figure 2.9

40. Criterion-Related Validity • Correlate a test score (predictor) with a performance measure; resulting correlation often called a validity coefficient • Predictive validity design • Time lag between collection of test data & criterion data • Test often administered to job applicants

41. Criterion-Related Validity (cont'd) • Concurrent validity design • No time lag between collection of test data & criterion data • Test administered to current employees, performance measures collected at same time • Disadvantage: No data about those not employed by the organization

42. Content-Related Validity • Demonstrates that content of selection procedure represents adequate sample of important work behaviors & activities or worker KSAOs defined by job analysis • I-O Psychologists can use incumbents/SMEs to gather content validity evidence

43. Construct-Related Validity • Investigators gather evidence to support decisions or inferences about psychological constructs • Construct - concept or characteristic that a predictor is intended to measure; examples include intelligence, extraversion, and integrity

44. A Model for Construct Validity Figure 2.10. A Model for Construct Validity

45. Construct Validity Model of Strength and Endurance Physical Factors Figure 2.11