Testing in Industrial & Business Settings

1. Testing in Industrial & Business Settings

2. How are people hired for a particular job or position? • Employment interview • Can be influenced by extraneous factor such as age, disability status, gender, race • Use of tests • When used in addition to interview & other sources of information, can improve chances of successful selection

3. Base Rates & Hit Rates • tests often used to make dichotomous decisions: • Pass/fail • Accept/reject • Hire/don’t hire • E.g., use of an 1100 score on the GRE to decide on whom to reject or whom to process further • This value is a cut-off score or a cutting score

4. The problem with cutting scores • Some people who have scores above the cutting score will do poorly • Some people who have scores below the cutting score would have done well

5. Hits & Misses Decision on basis of cutting score Accept Reject Success Actual Outcome Failure

6. Hit Rate • Percentage of cases in which a test accurately predicts success or failure

7. Hits & Misses Decision on basis of cutting score Accept Reject Success Actual Outcome Failure What is the hit rate?

8. Base Rate • The selection or pass rate obtained without using the test • E.g., using only undergraduate grades, letters of reference, etc. (no GRE), 70% of individuals selected are successful in completing their degree • If hit rate using the test is only 60%, test is of no value

9. False negatives & false positives • There are two types of “misses”: • False negative • Individuals not selected (because of a score below the cutting score) who would have been successful in job or program (e.g., GRE has high false negative rate) • Concluding on the basis of a test that a tumor is benign • Concluding on basis of a test that someone is not suicidal, when he/she actually is • False positive • Individuals selected for job or program who fail • Concluding on the basis of a test that someone has AIDS, when they do not have the disease

10. Decision on basis of cutting score Hire as pilot Reject Success Actual Outcome Failure What is more dangerous, false +ve or false –ve?

11. Decision on basis of cutting score Diagnose Aids Diagnose No Aids Has AIDS Actual Disease Status Doesn’t Have AIDS What is more dangerous, false +ve or false –ve?

12. Costs of False Negatives& False Positives • For certain decisions, false negatives will have a high cost • E.g., diagnosing a person as not having AIDS when she really does have it; that a person is not suicidal when he really is • For other decisions, false positives will have a high cost • E.g. hiring someone as a commercial pilot when he is likely to crash the plane

13. False negative, positive & criterion validity Success Actual Outcome False -ve False +ve Failure Cutting Score Test Performance

14. Changing Rate of False Negatives & False Positives by adjusting the cutting score • To reduce the number of false negatives, lower the cutting score • To reduce the number of false positives, raise the cutting score

15. Decision on basis of cutting score Diagnose Aids Diagnose No Aids Has AIDS Actual Disease Status Doesn’t Have AIDS Detection rate (sensitivity) = A/(A+B) = 8/10 Accuracy Rate = (A+D)/(A+B+C+D) = 83/100 (83%)

16. Taylor Russell Tables • helps evaluate validity of test in relation to amount it contributes beyond base rates • Requires • Definition of success -- patient lived, success on job, school • Determination of base rate • Definition of selection ratio -- % applicants selected • Validity coefficient • table gives likelihood that person selected on basis of test score will be successful

17. Taylor-Russell Table for a Base Rate of .60

18. Class Exercise • Suppose that you are director of graduate studies in psychology at Wilfrid Laurier University. You can choose 10 students for the graduate program. A total of 50 students apply to the program, and normally you would make your decision on the basis of undergraduate grades, letters of recommendation, work and volunteer experiences, and the like. In the past, about 60% of the students whom you admit complete their degrees. Up to this point, your program has not required applicants to submit GRE scores, but you are considering adding a GRE requirement for new applications, even though it correlates only about .25 with success in graduate school. • What is the base rate of success in this instance? • What is your success rate adding the GRE? • How many people will you reject that would probably have been successful in the program? (i.e., how many false negatives will there be?) • How many people will you accept who will likely not complete the program? (how many false positives will there be?)

19. Decision on basis of cutting score Accept Reject 30 Success Actual Outcome 20 Failure Base rate - .60 Selection ratio - .20 Validity coefficient - .25 10 40 50

20. Tests for Use in Industry • Bennett Mechanical Comprehension Test • Mechanical ability • Minnesota Clerical Test • Clerical skills • Computer Programmer Aptitude Battery • Computer programming skills

21. Integrity Tests • Also known as “honesty tests” • Paper-and-pencil inventories used for personnel selection to identify potentially dishonest or “counterproductive” employees • American businesses lose from $15- to $25-billion per year due to employee theft • 30% of all business failures attributed to employee theft

22. Personnel Selection Inventory (PSI) • Includes scales that measure: • Honesty • Drug avoidance • Tenure or job commitment • Safety • Work values • Supervision attitudes • Validity • Candidness • Accuracy

23. Sample Items Responses are recorded on 7-point scale ranging from “far more than average” (1) to “far less than average” (7)

24. Validity of PSI • In 1982 & 1983 a chain of home improvement centers screened more than 4,000 applicants with PSI to help reduce on-the-job theft and drug abuse. • PSI not used from 1979 to 1981 • “shrinkage” losses reduced from $7.5 million to $5.25 million • Number of employees terminated for using drugs at work reduced from 16 in 1981 to 1 in 1983

25. Measuring Characteristics of the Work Setting • Based on the premise that work environments will influence the productivity and satisfaction of individuals who work in those environments • Rudolph Moos developed scales to measure several different kinds of environments: • Work environments • Classrooms • Correctional institutions

26. Work Environment Scale • 10 subscales, organized into 3 dimensions: • Relationship • Personal growth • System maintenance & system change

27. Relationship Dimension • Involvement (extent to which employees are concerned about & committed to their jobs) • “people seem to take pride in the organization” • Peer cohesion (extent to which employees are friendly & supportive of one another) • “There’s not much group spirit” • Supervisor support (extent to which management is supportive of employees) • “supervisors tend to talk down to employees”

28. Personal Growth Dimension • Autonomy (extent to which employees are encouraged to be self-sufficient) • “people can use their own initiative to do things” • Task orientation (degree of emphasis on good planning, efficiency & getting the job done) • “this is a highly efficient, work-oriented place” • Work pressure (degree to which the press of work & time urgency dominate the job milieu) • “there is constant pressure to keep working”

29. System Maintenance & System Change Dimension • Clarity (extent to which employees know what to expect) • “things are sometimes pretty disorganized” • Control (extent to which management uses rules & pressures to keep employees under control) • “Supervisors keep a rather close watch on employees” • Innovation (degree of emphasis on variety, change, new approaches) • “New & different ideas are always being tried out” • Physical comfort (extent to which physical surroundings contribute to pleasant work environment) • “work space is awfully crowded”

30. Sample WES Profiles