Technology and the Practice of I/O Psychology

1. Technology and the Practice of I/O Psychology Nancy T. Tippins Personnel Research Associates, Inc. April 10,2003

2. Goals • Develop a framework for evaluating technological interventions in the practice of I/O psychology by • Exploring how technology affects the practice of I/O psychology today and in the future • Applying systematically a process for evaluation of a technological intervention

3. Agenda • Review examples of the use of technology in the practice of I/O psychology (25 minutes) • Example • Problems • Offer an approach for evaluating technology (10 minutes) • Discuss approach to specific content areas (15 minutes) • Share feedback on the approach (10 minutes)

4. Why Use Technology in I-O Psychology? • Speed • Volume • Distance/Geography • Availability • Just-in-time delivery • Costs • Integration with other systems • Innovative approaches

5. Areas of I-O Psychology • Training • Employee Development • Surveys • Multi-rater Feedback • Performance Management • Succession Planning • Team Building • Selection

6. Training Examples • Communications • Course Development • Training Delivery • Simulations and Work Samples • Assessment • Data Collection

7. Training Issues • Adapting computer-based training to different learning styles may be difficult. • Computer-based training may eliminate or reduce the opportunities for social interaction. • Computer-based training often requires personal discipline to complete training without frequent human interactions.

8. Employee Development Examples • Communications • Assessment • Data Collection and Record Keeping

9. Employee Development Issues • Some development programs are difficult to complete independently because they require • Discipline to complete activities independently • Introspective capability to assess own behavior • Acceptance of negative or critical feedback • Accurate and honest assessment • Technology cannot replace some activities, i.e., observation and feedback. • Some employees are afraid to be accurate. • Effect on relationships is not known.

10. Survey Examples • Communications • Administration • Analysis and Reporting

11. Survey Issues • The level of confidentiality that can be maintained is often unclear. • Responses • Reports • Ease of computer administration increases the risk of overuse of survey methodology.

12. Multi-Rater Feedback Examples • Communications • Administration • Analysis and Reporting

13. Multi-Rater Feedback Issues • Computer cannot anticipate reactions to information and assist the rated individual in working through strong emotional responses. • The level of confidentiality is unclear. • Overuse is a potential problem.

14. Performance Management Examples • Communications • Data Collection • Reports/Databases • Feedback and Coaching

15. Performance Management Issues • The level of confidentiality that can be maintained is often unclear. • Responses • Reports • Open, two-way communication may be difficult when participants are not face-to-face.

16. Succession Planning Examples • Communications • Data Collection • Reporting • Access to Data

17. Succession Planning Issues • Controlling access to succession planning data must be controlled in most organizations. • The structure of the organization must be known and maintained.

18. Team Building • Communications • Information • Relationship building • Problem solving

19. Team Building Issues • The effect of computer-based interactions on relationship building is unknown. • Opportunities for informal interactions may be lacking. • Confidentiality and privacy may be a concern of participants.

20. Selection Examples • Communications • Test preparation • Test administration session logistics • Interpretation guidelines • Administration instructions and rules • Feedback • Reports

21. Selection Examples • Test Administration • Previews • Instructions • Presentation • Scoring • Storage • Job Analysis Questionnaires • Validation Questionnaires • Novel Item Types

22. Selection Issues • Computer administration may change the construct measured or alter the appropriateness of the normative data. • Computer administration changes test taking strategies. • Test takers may not feel comfortable using computer despite assurances that computer ability required is minimal. • Reliability of timing and data recording are requirements that may difficult to achieve. • Security may be a problem particularly with internet/intranet distribution.

23. Selection Issues • ADA accommodations may be more difficult to provide. • Applicants may have potentially negative reaction to “low touch” selection processes. • Not all test takers have access to computer-based information about the testing process.

24. Advantages • Communications • Delivery • Timing • Costs • Speed • Integration with other systems • New ways of doing things

25. Concerns • Hardware and Software • Availability of equipment • Costs • Privacy • Appropriate software • Support/Maintenance • Skill requirements for I/O psychologists • Training required for user • Effects of technology on interpersonal interactions • Limitations on computer-based training

26. Concerns • User friendliness • User reluctance • User appropriateness • Need or desire for human sounding board • Anonymity, confidentiality, and privacy • Security • Ethics

27. Approach to Evaluating Technology • See Handouts

28. Example: Evaluating Technology • See Handout • 15 minutes • Discuss • Questions that should be added to the evaluation approach • Application for content area