Brian G. Whitaker Jason J. Dahling Samantha L. Chau Hsien-Yao Swee The University of Akron

1. Beyond measurement invariance: Using MIMIC modeling to investigate the effects of covariates on latent job performance ratings from differing ratings sources. Brian G. Whitaker Jason J. Dahling Samantha L. Chau Hsien-Yao Swee The University of Akron

2. Abstract This study examined whether a measure of task and contextual performance functions the same across both supervisors and subordinates (i,e., measurement invariant). As an extension, MIMIC modeling was used to examine potential covariates of task and contextual performance ratings (i.e., supervisor and subordinate age, gender, and tenure) on latent performance variability. Consistent with previous research, we found measurement invariance across self- and supervisor ratings. More importantly, MIMIC results showed supervisor and subordinate demographic variables systematically influenced latent performance variability despite measurement invariance over these rating sources. Implications for multi-source performance systems are discussed.

3. Introduction • Measurement invariance is most often tested using multi-group confirmatory factor analysis (MGCFA) • Internet and paper-and-pencil administration modes • gender • across cultures • Recently, differing rating sources • This study also seeks to extend the findings of the aforementioned literature by identifying correlates of latent job performance using a model known as a MIMIC model (multiple indicators, multiple causes)

4. MIMIC • extends MGCFA by analyzing the indirect effects of covariates in order to further analyze measurement invariance and population heterogeneity • an indirect effect for a given covariate would indicate that mean differences for latent performance variability were found as a function of differing levels of that covariate • performance ratings may be influenced by and correlated with rater demographic variables (ie, sex, race, tenure)

5. Summary • To date, no one has assessed measurement invariance of job performance ratings across rating sources and the impact of covariates on latent job performance variability simultaneously, using the same dataset. For this reason, the present study seeks to contribute to and extend the current literature by 1) investigating the invariance of contextual and task performance ratings across rating sources, and 2) examine the effects of covariates on latent performance variability using a using SEM, specifically, MIMIC modeling.

6. Subordinate Gender Subordinate Age Subordinate Tenure Task Performance Contextual Performance Manifest Indicators 1-7 Manifest Indicators 1-20 Figure – MIMIC Model – Subordinate Ratings

7. Supervisor Gender Subordinate Gender Supervisor Age Subordinate Age Supervisor Tenure Subordinate Tenure Task Performance Contextual Performance Manifest Indicators 1-7 Manifest Indicators 1-20 Figure – MIMIC Model –Supervisor Ratings

8. Method • Participants - One hundred seventy supervisor-subordinate dyads • Measures - Williams and Anderson’s (1991) seven-item measure of in-role behavior (task performance); Williams and Anderson’s (1991) 20-item measure of organizational citizenship behavior (contextual performance). • Procedure - As part of a larger study, subordinate participants completed measures designed to assess their perceptions of their job performance and provide relevant demographic information. Subordinates completed a consent form allowing their supervisors to be contacted. The supervisor survey assessed employee task performance and contextual performance as well as supervisory demographic information. Supervisors then mailed the completed surveys back to the researchers.

9. Results – MIMIC Model – Subordinate Ratings • Freeing of two item variances, job performance ratings were invariant across supervisor and self-ratings (Table 1) • First MIMIC model analyzing the effects of rater gender, age, and tenure on self-ratings of job performance (Table 2) • Relationship between subordinate gender and latent task performance variability was marginally significant • Subordinate tenure was positively related to latent task performance ratings; marginally significant for latent contextual performance variability

10. Results – MIMIC Model – Supervisor Ratings • Analyzing the effects of rater and ratee gender, age, and tenure on supervisor ratings of job performance(Table 2) • supervisor gender was correlated with latent performance variability only for contextual performance ratings • subordinate age was negatively related to latent performance variability for the contextual performance • effects for both supervisor and subordinate tenure were found across both performance dimensions

11. Table 1 - Tests of measurement invariance across supervisor and self ratings Note: *p < .05. CFI = Comparative Fit Index, RMSEA = Root Mean Square Error of Approximation, SRMR = Standardized Root Mean Squared Residual.

12. Table 2 - Relationships between covariates and latent performance constructs across supervisor and self ratings Note: *p < .05. For the gender variable, males were coded as 1, females coded as 2.

13. Results • MIMIC model analyzing the effects of rater gender, age, and tenure on self-ratings of job performance • age was not associated with latent performance variability for either performance dimension • subordinate gender and latent task performance variability was marginally significant • positive relationship with latent task performance ratings; marginally significant for latent contextual performance variability

14. Discussion • Respondent characteristics may systematically influence the responses made to the measurement device differentially across administration mode, despite measurement equivalence • supervisors seem to systematically rate older workers lower on performance elements not associated directly with the task • female supervisors tend to rate contextual performance higher than male supervisors • experienced employees receive higher ratings from their supervisors and rate themselves higher • tenure of the supervisor was related to higher ratings of both task and contextual performance

15. Implications • Examine covariates of both task and contextual performance separately. Our findings indicate the future research should make this distinction due to the differing sources of variability for both types of performance • More demographic variables influence the latent variability of contextual performance ratings than task performance. Raters may differ according to implicit rating policies. • Rater training in organizations should take into account the influence of these demographic variables on ratings.

16. Yao presenting at SIOP