STRUCTURAL EQUATION MODELING AND META-ANALYSIS: A MARRIAGE MADE IN ...

1. STRUCTURAL EQUATION MODELING AND META-ANALYSIS:A MARRIAGE MADE IN ... Ronald S. Landis Illinois Institute of Technology

2. 1+1=?? • Peanut Butter & Chocolate • Food & Wine • Movies • LeBron James & Dwayne Wade • The Whole is More, Equal to, or Less than the Sum of Its Parts

3. Plan for Today • Meta-Analysis (MA) FAQ • Structural Equation Modeling (SEM) FAQ • MASEM • Research Questions & Focus • Steps/Decisions/Frustrations • Odds & Ends

4. MA FAQ • Research Questions • Summarize available (published & unpublished) evidence • Estimate degree of relation between variables • Identify conditions in which effect differs

5. MA FAQ • Foundational Issues & Pragmatic Concerns • Starting with a clear purpose • Why not just collect and code everything? • “HARK”ing • Nature of available evidence • Samples, research designs, measures • When is there enough information? • k’s and n’s • Adequate construct development • What am I looking for and how do I know if I have found it? • File Drawers • What if I’m missing something?

6. SEM FAQ • Research Questions • To what degree are a set of indicators associated with a given set of latent variables (CFA)? • Are hypothesized causal relations between variables consistent with collected data (PA/Full SEM)? • Are there systematic patterns of change across time (LCM)? All require a priori specification of a particular model (or set of models)

7. SEM FAQ • Foundational Issues & Pragmatic Concerns • Starting with a clear purpose • Why not just collect some data and start testing models? • “HARK”ing • Nature of available evidence • Samples, research designs, measures • When is there enough information? • n’s and identification • Adequate construct development • What am I looking for and how do I know if I have found it? • Everything relevant is included • What if I have omitted some important variables?

8. MASEM • Possible Combinations • SEM used for MA (Cheung, 2008) • MA as input for SEM [TODAY] • Technical/Practical Focus • What are the underlying mathematical/statistical elements? • How do I undertake these analyses in my own work? [TODAY] • Primary Questions • Can I Integrate SEM and MA? • YES • How Do I Integrate SEM and MA? • HOPEFULLY A CLEARER UNDERSTANDING • Should I Integrate SEM and MA? • THE $64,000 QUESTION – IT DEPENDS

9. General Steps Proposed by Viswesvaran & Ones (1995) • ID Core Constructs and Relations • ID Measures Used to Assess Each Construct • Obtain Relevant Studies • Conduct Meta-Analysis • Test Measurement Model • Estimate Construct Correlations • Apply SEM to Test Specified Relations

10. Two-Stage Structural Equation Modeling (TSSEM)Proposed by Cheung & Chan (2005) STEP 1 • Test for Homogeneity of Correlations/Covariances • Using multiple groups CFA • Calculate Pooled Correlation/Covariance Matrix • Create Asymptotic Covariance Matrix STEP 2 • Apply SEM to Test Specified Relations BENEFIT • Accounts for second order sampling error, leading to more accurate chi-square and fit indices

11. General Approach – Model Testing • Confirmatory Factor Analysis • Same measures/indicators across studies • Covariances • Path/Structural Models • Different measures/indicators across studies • Correlations

12. Logic • First Step • Test of homogeneity • Build input matrix • Second Step • Model testing

13. An Example –Earnest, Allen, & Landis (2011) • Student milestone project • Interest in Realistic Job Previews (RJPs) • How do we build on current MA’s? • Increased number of studies • Increased number of moderators • Conducted Meta-Analysis • Consideration of potential mediators via SEM

14. An Example

15. Example Matrix Construct the correlation matrix

16. Issues in Constructing the Correlation Matrix • What about blank cells? • Listwise deletion? • Previous meta-analyses • Studies included in current meta-analysis • Meta-analytic values from ‘outside’ studies • Different sample sizes in cells • Which one do we use? • Illogical values/npd matrices • What if my matrix can’t occur? • Presence of moderators • What do I do if I have heterogeneity in the correlation matrices? • Correlations instead of covariances • Can I still run SEM?

17. Odds & Ends • But… • Causality and MASEM make strange bedfellows • Yes, but this is not unique to MASEM • Lack of comfort with combining r’s from different studies into SEM analysis • Why? • What if latent variables differ across primary studies • Problem with MA, not MASEM

18. A Few References • Burke, M.J. & Landis, R.S. (2003). Methodological and conceptual challenges in conducting and interpreting meta-analyses. In K.R. Murphy (Ed.), Validity Generalization: A Critical Review (pp. 287-309). Mahwah, NJ: Erlbaum. • Cheung, M.W.L. (2009). TSSEM: A LISREL syntax generator for two-stage structural equation modeling (Version 1.11) [Computer software and manual]. Retrieved from http://courses.nus.edu.sq/course/psycwlm/internet/tssem.zip. • Cheung, M.W.L. & Chan, W. (2005). Meta-analytic structural equation modeling: A two-stage approach. Psychological Methods, 10, 40-64. • Cheung, M.W.L. & Chan, W. (2009). A two-stage approach to synthesizing covariance matrices in meta-analytic structural equation modeling. Structural Equation Modeling, 16, 28-53. • Christian, M. S., Bradley, J.C., Wallace, J.C., & Burke, M.J. (2009). Workplace safety: A meta-analysis of the roles of person and situation factors. Journal of Applied Psychology, 94, 1103-1127.

19. A Few References • Colquitt, J. A., LePine, J. A., & Noe, R. A., toward an integrative theory of training motivation: A meta-analytic path analysis of 20 years of research. Journal of Applied Psychology, 85, 678-707. • Earnest, D.R., Allen, D.G., & Landis, R.S. (2011). A meta-analytic path analysis of the mechanisms linking realistic job previews and turnover. Personnel Psychology, 64, 865-897. • Klein, H.J., Wesson, M.J., Hollenbeck, J.R., Wright, P.M., & DeShon, R.P. (2001). The assessment of goal commitment: A measurement model meta-analysis. Organizational Behavior and Human Decision Processes, 85, 32-55. • Robbins, S. B., Oh, I., Le, H., Button, C. (2009). Intervention effects on college performance as mediated by motivational, emotional, and social control factors: integrated meta-analytic path analyses. Journal of Applied Psychology, 94, 1163-1184. • Viswesvaran, C. & Ones, D.S. (1995). Theory testing: Combining psychometric meta-analysis and structural equations modeling. Personnel Psychology, 48, 865-885.