Competency Modeling & Situational Judgment Interview

1. Competency Modeling & Situational Judgment Interview Presented By Marina Mihalevsky & Kristin Olson Bassett USD, Personnel Commission Scott Letourneau, NEOGOV, Inc.

2. PRESENTATION OBJECTIVE To gather information on use and development of competency modeling for enhancing assessment and organizational cohesiveness and efficiency. Bassett USD Personnel Commission

3. SESSION OUTLINE • Pre-Presentation Quiz! • Presentation Objective • History: Emergence of Competencies • Competencies Defined • Competency Modeling • Developmental Steps for Competency Models • Application of Competencies: Case Study-Situational Judgment Interviews • Best Practices • Limitations • Enhancing Competencies with Technology

4. QUIZ: QUESTION ONE • What types of organizations are using competencies? • Small public agencies • Large public agencies • Small private companies • Large private companies Bassett USD Personnel Commission

5. QUIZ: ANSWER ONE • The answer is ALL types of organizations can use and benefit from using competencies • 75% (Cook & Bernthal, 1998, based on a survey of 292 organizations) • 80% (American Compensation, 1996, based on a survey of 426 organizations) of responding companies have some competency-driven applications currently in place. Bassett USD Personnel Commission

6. QUIZ: QUESTION TWO 2. What areas in an organization can competencies impact? (a) Job Descriptions (b) Recruitment (c) Selection Tools (d) Training Programs (competency enhancement/orientation training) (e) Performance Evaluation & Management (f) Career Planning & Career Consulting (g) Professional Development (h) Succession Planning & Promotion Bassett USD Personnel Commission

7. QUIZ: ANSWER TWO • The answer is ALL – competencies can impact and cohesively connect almost all areas of an organization. Additional areas include: • Mission statement/Vision statement • Organizational/departmental/personal goals • Enhance a culture of teamwork Bassett USD Personnel Commission

8. QUIZ: QUESTION THREE 3. What is the current satisfaction with competency applications? (a) Organizations are dissatisfied or very dissatisfied (b) Organizations are satisfied or very satisfied Bassett USD Personnel Commission

9. QUIZ: ANSWER THREE More experienced, skilled implementers were more satisfied (30%) with the outcomes of their projects (Schoonover et al, 2000) • Used more multi-dimensional criteria in their models (outcome/results criteria) • Implemented competencies across more organizational levels and across more functions • were designing and implementing more comprehensive performance and development feedback systems. • Were enabling their competency applications through technology to a greater extent Implication: The more areas competencies impacted in the organization the GREATER the satisfaction.

10. HISTORY: Emergence of Competencies • 1970s: “Testing for Competence Rather than Intelligence” (McClelland,1973) • Competency:“an underlying characteristic of a person which enables them to deliver superior performance in a given job, role, or situation” • Can be learned and developed over time. • Implication: If competencies are made visible and training is accessible, individuals can understand and develop the required level of performance.

11. HISTORY:Emergence of Competencies • 1980s: • Boyatzis (1982) - Competency model assessing manager’s effectiveness • Zemke (1982) - Applying job competencies to training • 1990s • Prahalad and Hamel (1990) & Lawler (1994) • Competencies allow an organization to achieve and sustain a competitive advantage • 2000s: Standardization for CM recommended (SIOP) • Shippmann et al. (2000) - Evaluating JA and CM on a 10-dimension Level of Rigor Scale

12. DEFINITION: What is a competency? • “A combination of motives, traits, self-concepts, attitudes… content knowledge, or cognitive behavior skills; any individual characteristic that can be reliably measured…and… shown to differentiate superior from average performers” (Spencer, McClelland, & Spencer, 1994). • “A measurable pattern of knowledge, skills, abilities, behaviors, and other characteristics…an individual needs to perform work roles…successfully” (Office of Personnel Management). • “…observable performance dimensions, including individual knowledge, skills, attitudes, and behaviors, as well as collective team, process, and organizational capabilities, that are linked to high performance, and provide the organization with sustainable competitive advantage” (Arvy & Orth, 1999).

13. DEFINITION:Competency Definition Components • Knowledge and skills associated with current job performance • Work habits or manner of conducting oneself that enables effective work performance (Marrelli, 2000; Spencer et al,1994) • Process capabilities that enhance organizational or business performance (Arvy & Orth, 1999) • New ways of thinking or behaving that provide distinctive competitive advantage (Arvy & Orth, 1999) • Measurable and observable • Implication: Competencies move beyond the very specific (minutia) job requirements associated with tradition job descriptions

14. DEFINITION: Core VS. Technical Competencies • Core Competency Definition: Competencies that may be shared across a job family or all organization members. • Technical Competency Definition: Competencies that are unique to classifications or job families. Bassett USD Personnel Commission

15. COMPETENCY MODELING:What is a Competency Model? A process of systematically identifying a set of competencies that are necessary for successful performance in a particular job or job family. • Behavioral indicators associated with performance on each competency are specified. Bassett USD Personnel Commission

16. COMPETENCY MODEL FRAMEWORK Indicator (1) Competency 1 Indicator (2) Indicator (3) Competency 2 Competency 3 Bassett USD Personnel Commission

17. COMPETENCY MODELING:How is it different than Job Analysis? • CAMP A: Different • Less reliant on job analytic information • Prescriptive VS Descriptive • Broader focus and alignment with organizational strategy and goals • CAMP B: Essentially the same • Worker-oriented type of job analysis • Face Validity (association with ‘competent’ and ‘competence’) Bassett USD Personnel Commission

18. DEVELOPING A COMPETENCY MODEL (1) Step One: Definite the Objective (2) Step Two: Rally Support (3) Step Three: Information Gathering (4) Step Four: Developing a Competency Bank-Write the Competencies (5) Step Five: Developing Competency Mastery Levels (6) Step Six: Data Gathering, Collection, & Analysis (7) Step Seven: Create Competency Profiles

19. STEP ONE: Define the Objective What is the purpose? • Selection Instruments • Performance Evaluation • Training (orientation/developmental) • Career & Professional Development • Supervisor guided or employee initiated • Succession Planning: Culture of Continuous Development and Learning Bassett USD Personnel Commission

20. STEP TWO: Rally Support Utilize a range of communication mediums to gain support for the project and its implementation, examples: • Newsletters • Meetings It is important that management, different departments, staff level employees, and unions, etc., are all included in the development process. Bassett USD Personnel Commission

21. STEP THREE: Information Gathering & Developing a Competency Bank • Identify Relevant Competencies Sources of Information: • Organizational Mission Statement/Vision Statement/Goals • Literature Review (organizational documents and topical psychology research) • Models from other organizations • Survey organizational members Bassett USD Personnel Commission

22. STEP FOUR: Developing a Competency Bank-Writing the Competencies • Define Competency • E.g. Observable, measurable pattern of behavior and capability that is necessary for successful performance. • Competency:Oral Communication The capacity to effectively verbally communicate ideas and information to others through attention to grammar, tone, inflection, and speech formality.

23. STEP FIVE: Developing Competency Mastery Levels • What is a Mastery Level? • Describe the degree of competency • Definitions provide a standard that is not changed, while the examples (i.e, task statements) can be modified and made job specific. E.g. Competency: Goal Setting Level 1 Level 2 Level 3 Adheres to goals and deadlines set by supervisor Sets goals and priorities for own work to ensure deadlines are met. Sets goals and priorities for own work and coordinates activities and time-lines with/for others.

24. Tips on Developing Mastery Level Definitions • Levels of Competency Standards (5-levels) • Scope • Context • Responsibility/Accountability • Management of Resources • Bloom’s Levels of Taxonomy • Work level definitions Bassett USD Personnel Commission

25. COMPETENCY MODEL FRAMEWORK WITH MASTERY LEVELS Competencies Mastery Level Behavioral Indicator Competency 1 Level 1 Indicator (1) Level 2 Indicator (2) Level 3 Indicator (3) Bassett USD Personnel Commission

26. STEP SIX: Data Gathering • Some commonly used data gathering techniques to consider: • Behavioral Event Interview (adaptation of a critical-incident technique) • Focus Groups • Survey Bassett USD Personnel Commission

27. DATA GATHERING: Survey Design Common Survey Questions: • How important is the competency to a given job? • How well does a given competency distinguish between high and low performers? • Is the competency required at job entry? • Is there a need for training? Bassett USD Personnel Commission

28. DATA GATHERING: Survey Administration • Survey Administration • SME Characteristics • Experienced SMEs are better able to discriminate between competency requirements across jobs (Lievens et al., 2004) • Job Descriptor Content • Job Description • Organizational Chart • Training Objectives • Performance Evaluation Forms • Organizational Mission Statement & Goals

29. DATA GATHERING: Data Collection & Analysis • Set Timeline & Follow-Up • Set criteria for minimum number of raters required to rate a competency “crucial” • Determine how you will figure out required levels of mastery (average, mode, authority) • Factor Analysis to determine competency clusters Bassett USD Personnel Commission

30. STEP SEVEN: Create Competency Profiles for Jobs Job Title: Personnel Analyst/HR Specialist Competency: Information Exchange (Definition) Mastery Level: (4) Has access to privileged information. Provides information that is timely, credible, and appropriate to the needs of others. Uses multiple communication channels as necessary to ensure communication occurs. Competency: Conflict Management & Negotiation (Definition) Mastery Level: (3) Maintains composure when faced with an escalating conflict, and facilitates a discussion among the involved parties that addresses the issues underlying the conflict.

31. ORGANIZATIONAL MODEL FRAMEWORK Competency Cluster: COMMUNICATION Bassett USD Personnel Commission

32. COMPETENCY-BASED APPLICATIONS CASE STUDY: SITUATIONAL JUDGMENT INTERVIEW Bassett USD Personnel Commission

33. CASE STUDY:Applying the Competency Modeling to Situational Judgment Interviews • What is a Situational Judgment Interview? • It is an interview where specific job situations are posed to applicants and a rater assigns a score to the answer, based on a continuum of excellent to poor responses. • “Examiner has [little] discretion in the selection of test questions or the evaluation of the answers” (Ed. Code) Bassett USD Personnel Commission

34. CASE STUDY:Applying the Competency Modeling to Situational Judgment Interviews • Steps for Developing a SJI • Information Gathering (Critical Incidents) • observation/interviews/surveys • Write the Items & Responses • Have SMEs edit the items and responses • Face validity editing • Link the test items to the competencies in a job description --SMEs • Content validation • Determine Scoring Methodology Bassett USD Personnel Commission

35. CASE STUDY:Applying the Competency Modeling to Situational Judgment Interviews • Validating a Situational Judgment Interview: Using Competency modeling • Performance Evaluation (Selection Evaluation) • Allows for criterion-related validity estimates Bassett USD Personnel Commission

36. BEST PRACTICES FOR IMPLEMENTING COMPETENCY-BASED APPLICATIONS • Ensure that the competency initiative supports the organizational strategy. • Focus on integrating competencies with all HR processes. • Focus on organization-wide implementation and ongoing evaluation. Bassett USD Personnel Commission

37. Barriers to Successful Development & Implementation • Lack of Expertise • Limited Support • Competing Priorities • Lack of Staff Resources • Lack of Fiscal Resources • No set methodology or standard currently exists for developing a competency model, like there is for job analyses, BUT this can be overcome with structured data gathering that adhere to job analytic techniques

38. Using Technology to Enable Competencies • Competency Anatomy • Competency Bank • Create Selection Tool • Create Performance Assessments • Conduct Criterion Validation • Measure Training Effectiveness • Track Certification and Training • Organize Succession Planning

39. Class Specification Performance Assessment Competency Training Selection Tool Written Item Oral Interview Item Competency Anatomy

40. Competency Bank Competency Bank Core Comp C1 C2 TI Q1 Train 1 C3 Mastery Level T1 T2 TI Q2 Train 2 BI1 BI2 Competency Bank contains: • All Core & Technical Competencies • Selection Tool Test Items • Required Certificates and Trainings based on each Mastery Level • Behavioral Indicators based on each Mastery Level • TI Q: Test Item • Train: Required Trainings • BI: Behavioral Indicator

41. Create Selection Tool How to Create • Decide on selection tool (written, oral, T & E, etc.) • Based on the Class Specification competencies and corresponding mastery levels, go into the competency item bank and select the appropriate test questions Core Comp 1 TI Q1 Mastery Level 3 TI Q2 Selection Tool: C1-ML3-TI1 C1-ML3-TI2 C1-ML4-TI1 C1-ML4-TI2 Core Comp 2 TI Q1 Mastery Level 4 TI Q2

42. Create Performance Assessment Core Comp 1 Core Comp 2 Mastery Level 3 Mastery Level 4 How to Create • Based on the Class Specification competencies and corresponding mastery levels, go into the competency item bank and select the appropriate behavioral Indicators BI1 BI2 BI1 BI2 Performance Assessment: C1-ML3-BI1 C1-ML3-BI2 C1-ML4-BI1 C1-ML4-BI2

43. Selection Tool Performance Assessment Core Comp 1 Core Comp 1 TI Q1 Mastery Level 4 Mastery Level 4 TI Q2 BI1 BI2 BI1 BI2 Criterion Validation Goal is to obtain validity estimates by examining the relationship between selection tool test items with the behavior indicators of the on the job performance results. Report would show the relationship between competency scores on both selection tool and performance assessment through regression analysis

44. Class Specification C1 – ML3 Performance Assessment Training Competency C1 – ML4 Mastery level should increase Training Evaluation Goal is to validate that trainings increase the on the job behavior indicators for required mastery levels. Report would show change results of behavioral indicators for particular mastery levels compared with the completed training.

45. Performance Assessment Indicators • Criterion Validation • Selection Tool Test Item results CORRESPOND with On the Job Behavioral Indicators in the Performance Assessment • Training Validation • On the Job Mastery Level Behavior Indicators in the Performance Assessment INCREASE as a result of attending Trainings

46. Succession Planning HR Director C1 – ML 5 C2 – ML 5 T1 – ML 5 HR Specialist Personnel Analyst II C1 – ML 4 C2 – ML 3 T1 – ML 5 C1 – ML 3 C2 – ML 3 T1 – ML 5 • Challenge • How to track what competencies your current employee base posses • How to structure create and define succession plan (needs analysis in competencies, training, and certifications) HR Admin I HR Admin I C1 – ML 1 C2 – ML 2 T1 – ML 2 C1 – ML 2 C2 – ML 2 T1 – ML 2

47. HR Director C1 – ML 5 C2 – ML 5 T1 – ML 5 HR Specialist Personnel Analyst II C1 – ML 4 C2 – ML 3 T1 – ML 5 C1 – ML 3 C2 – ML 3 T1 – ML 5 HR Admin I HR Admin I C1 – ML 1 C2 – ML 2 T1 – ML 2 C1 – ML 2 C2 – ML 2 T1 – ML 2 Succession Plan • 1) Report of needs analysis of comps and MLs • 2) Comps and MLs are tied to trainings and certifications • Results are: • HR Specialist needs • C1-ML4-ML5, which requires training A • C2-ML3-ML5, which requires training C and D • PA II needs • C1-ML3-ML5, which requires training A and B • C2-ML3-ML5, which requires training C and D ** Competencies allow you to do true succession planning, above an beyond recruitment planning

48. Recap Criterion Validation • Effectively identify selection tools and items are / are not good indicators of actual job performance (TI : BI) ** Employees do not need to know “how” to do criterion validation, they just run the reports and the statistics determine the validity results. Training Effectiveness • Accurately identify which agency trainings have a positive / negative / no impact on employee growth and competency enhancement (BI from performance Assessment : Train) Succession Planning • Quickly establish a succession plan detailing what competencies, trainings, and certificates are required for every employee to move to every position within the agency (i.e., a Clerk Typist I to become Director of Public Works and determine the path and timing of how they can get there)

49. Strategic Organizations Recruitment, Selection/Testing, Validation, Planning = Strategic Organization Recruitment Selection/ Testing Performance Evaluations Training / Certifications Succession Planning Results: Time Effort Cost Service Satisfaction Retention

50. Thank You! Marina Mihalevsky & Kristin Olson Bassett USD, Personnel Commission Scott Letourneau President NEOGOV