M21 : Assessment in the Workplace

1. M21 : Assessment in the Workplace Selection Validation, Job Analysis and Job Analysis Techniques

2. Overview • Job Analysis • What is it? • Why do it ? • Job Analysis Techniques • description • advantages & limitations • Selection Validation • classical model (local criterion-related) • content related validation (global procedures, job element method) • Synthetic validation • Bayesian Model

3. Purpose of a selection process • Predicting future job performance • gather job relevant information about the candidate • discriminate between candidates in their ability to do the job • assess ‘person-environment’ fit • PR exercise - selling the organisation to the candidate • Empowerment of employees • Demonstrate adherence to employment legislation

4. To determine whether one person will be ‘better’ in job ‘x’ than another we must have criteria relevant to job ‘x’ on which to judge their suitability. Job Analysis provides a way of determining what KSA’s are important (essential/desirable) Job Analysis... ‘any systematic procedure for obtaining detailed and objective information about a job’

5. Other functions of Job Analysis Notjust for selection ….. • Enables ‘problems’ to be solved on the basis of data & evidence, rather than subjective speculation or history • Gives insight into why problems may occur/have occurred • Rapid developments in technology have resulted in constant changes in work (individual & group level) • Can identify learning and training needs of current employees JA

6. Choosing a Job Analysis Method Aim is to analyse the job in such a way that the functions, outputs or dynamics between specific tasks are not artificially distorted. Threats to Accuracy • Time Determined Changes • Person Determined Changes • Situation Determined Changes

7. Dimensions along which JA techniques differ • Orientation (worker- vs. task-oriented) • Quantification (qualitative vs. qualitative data) • Structure (open-ended vs. closed) • Packaging (off-the-shelf vs. DIY) • Sophistication (e.g. PAQ vs Observation) • Proximity to Jobs (remote from- vs close to/part of job being studied) • Applicability (broad vs narrow) • Sensitivity (adaptable or inflexible) • What kind of information is needed ? • What resources are available ?

8. JA Methods • Observation • Self-Description (diaries) • Interviews • Critical Incident Technique • Repertory Grid • Checklists/Inventories • Hierarchical Task Analysis • Position Analysis Questionnaire

9. Observation • Type of job? • What to record? • Advantages/Disadvantages • (+’s) yield numerical ratings; potential for accurate, objective record • (-’s) labour intensive, Hawthorne effect; a-priori categorisation

10. Self-Description (Diaries) • Suitable for most jobs • What to record? • Advantages/Disadvantages • +’s : Potential for accurate record; ‘inside workings’ • -’s : Objectivity? Drop-out rate; A-priori categorisation

11. Interviews • Suitable for all jobs • Identifying stakeholders • Interview Structure • Advantages/Disadvantages • +’s : flexible, less intrusive • - ‘s : impression management, requires familiarity with the role, accuracy rests on interviewee’s recall; interviewer-interviewee interaction

12. Interviews II : Critical Incident Technique(Flanagan, 1954) • Incident has to be observable, so that inferences can be made about the person’s response • The incident must be critical • Decide a ‘critical’ incident • Determine what was the background to the incident • What did the person do that was very effective/very ineffective • Why does that response make it very effective/very ineffective • Advantages/Disadvantages • +’s : flexibility, psychological fidelity, amount of information • -’s : can only be used where incumbent/job exist; where to stop?!

13. Hierarchical Task Analysis(Annett & Duncan, 1971) • Principles • tasks can be defined in terms of objectives/end products • aims to identify the plan (schema) the job holder uses to achieve the objective • the plan is organised into a hierarchical series of sub-plans and sub-goals • the behaviour that is observed when analysing a task = ‘operations’; each sub-plan has an operation (and sub-operations) • the analysis will yield ‘task statements’ (activity verb, performance standard and conditions/context of operation) • Advantages • skilled, semi-skilled manufacturing type jobs with only a limited number of ‘right ways’ • can only be used for jobs in existence; what level of detail to go to?

14. Position Analysis Questionnaire • Information input • Mental processes • Work output • Relationships with other persons • Job context • Other job characteristics • Advantages/Disadvantages • +’s : job doesn’t have to exist (or have incumbent); numerical data, minimal resources required. • -’s : limitations of quantitative approach.

15. Selection Validation How to determine whether the KSA’s identified by the job analysis (and assessed by particular selection techniques) are appropriate N.B. Face, Content, Criterion and Construct Validity 1. Classical (Local Criterion Related) Model • conduct JA • choose criterion • choose selection techniques • pilot selection process • data analysis (selection data as predictors of ‘x’, differential validity ?) • revise selection process (if necessary)

16. Problems with ‘classical’ model • Criterion data unsuitable and/or unavailable • Inadequate number of employees performing the same job • Restriction of range (through pre-selection) • Extraneous (individual/organisational) variables distorting predictive validity • Takes too long to obtain results (jobs change, selection situation changes, organisational roles change) • Number of additional problems if use concurrent validation (current employees) as opposed to predictive validation (applicants) - attrition, varying skill/experience, future jobs?

17. Alternatives Content Related Validation • Global Procedures : what differentiates one job from another job • Job Element Method : only those elements that differentiate between best and worst employees Synthetic Validation • detailed JA - identify job elements and weights • for each element : analysis & empirical study of selection technique intended to measure it • determine validity of selection process by combining validity coefficients identified x weighting : Primoff (1958) ‘j-coefficient’ Bayesian Model (Schmitt & Hunter, 1977) • previous validation; estimate variability in error of validity coefficients reported.