Daniel Caro and Andrés Sandoval-Hernandez CIES 2012 Annual Conference

1. A New Look at the Evaluation of Sociological Theories in International Large Scale Educational Assessments Daniel CaroandAndrés Sandoval-Hernandez CIES2012 Annual Conference April 25, San Juan, Puerto Rico

2. Rationale • Two important criticisms of LSA studies are • Studies lack a solid theoretical basis • This affects the selection, operationalization and explanation of the relations between variables • e.g., extended use of SES indicators • Techniques for developing and testing theories (EFA and CFA) have several limitations, for example: • EFA lacks model fit indices • CFA is too restrictive • This study addresses these criticisms

3. Purpose • Evaluate sociological theories of economic, cultural, and social capital postulated by Bourdieu, Bernstein, and Coleman • Investigate if theoretical models are reflected by the data of international student assessments • Demonstrate advantages of ESEM • Explore factor structure • Evaluate model fit • Test MI across countries

4. Theoretical Framework • Economic capital (Bourdieu, 1983) • Economic capital is defined as the command individuals have over economic resources • The concept is commonly understood as exchange values, like income and assets that can be easily transformed into cash • Cultural capital (Bernstein, 1971; Bourdieu, 1983) • Cultural capital is accounted by the cultural long-lasting dispositions embedded in the human mind and body, as well as in cultural goods and educational credentials • Cultural capital can appear in three states: objectified, institutionalized, and embodied

5. Theoretical Framework • Social capital (Coleman, 1988) • Social capital are all those aspects of the social structure that can be used by actors as resources to achieve their interests • Family social capital is represented by family relations that enhance the transmission of other structural resources (e.g. dispositions towards culture)

6. EFA and CFA • In general, • EFA for generating theory • CFA for testing theory • CFA limitations (Marsh, 2009, 2010): • The zero cross-loading restriction • Statistically, leads to inflated interfactor correlations • Also theoretically problematic , e.g. number of books reflects both cultural capital and economic capital • Often well-defined EFA structures are not supported by CFA • In practice researchers perform exploratory CFA

7. Exploratory Structural Equation Modeling (ESEM) • Marsh et al. (2009, 2010) introduced ESEM, an integration of EFA and CFA • Essentially, ESEM is EFA with • Standard errors • Model fit indices • MI tests • ESEM is appropriate if model fit is better than in CFA • To our knowledge, ESEM has not been yet applied to sociological theories

8. Data • PIRLS 2006 • 42 educational systems • 21,000 students (500*42) • 36 items from parent questionnaire • PISA 2009 • 14 countries • 7,000 students (500*14) • 66 items from student and (optional) parent questionnaire

9. Analytical Strategy • EFA with all items • Selected items (indexes) for 3 factors • 14 in PIRLS 2006 • 12 in PISA 2009 • CFA and ESEM with selected items • Model fit • Interfactor correlations • Cross-loadings • Measurement invariance (MI) tests by country’s HDI

10. PIRLS 2006: CFA results CFI= 0.77 TLI= 0.71 RMSEA= 0.07 0.30 0.73 0.19 Cultural Capital Economic Capital Social Capital Academic communication Parental attitudes to reading Books at home Parental education Childrens’ own room Visits to library / booksstore Literacy support activities Communication & Interaction Cultural communication Parents' reading at home Children's book Parental ocupational status Financial status Parents' reading for enjoyment

11. PIRLS 2006: ESEM results CFI= 0.77 0.96 TLI= 0.71 0.93 RMSEA= 0.07 0.03 0.30 0.24 0.73 0.41 0.19 0.08 Cultural Capital Economic Capital Social Capital Academic communication Parental attitudes to reading Books at home Parental education Childrens’ own room Visits to library / bookshop Literacy support activities Communication & Interaction Cultural communication Parents' reading at home Children's book Parental ocupational status Financial status Parents' reading for enjoyment

12. PIRLS 2006: CFA and ESEM factor loadings

13. PISA 2009: CFA results CFI= 0.87 TLI= 0.83 RMSEA= 0.07 0.36 0.68 0.28 Cultural Capital Economic Capital Social Capital Academic communication Cultural possessions Parents' reading motivation Books at home Parental education Visits to library / bookstore Literacy support activities Communication & Interaction Cultural communication Home educational resources Parental ocupational status Household overcrowding

14. PISA 2009: ESEM results CFI= 0.87 0.96 TLI= 0.83 0.93 RMSEA= 0.07 0.05 0.36 0.15 0.68 0.38 0.28 0.17 Cultural Capital Economic Capital Social Capital Academic communication Cultural possessions Parents' reading motivation Books at home Parental education Visits to library / bookstore Literacy support activities Communication & Interaction Cultural communication Home educational resources Parental ocupational status Household overcrowding

15. PISA 2009: CFA and ESEM factor loadings

16. Measurement Invariance Tests Two groups of countries PIRLS 2006: low/medium HDI versus high HDI PISA 2009: high HDI versus very high HDI

17. Discussion • Evaluated theories of economic, cultural, and social capital are reasonably reflected by the data • ESEM performs better than CFA to capture these theories • Better model fit • Greater discriminant validity (lower interfactor correlations) • Cross-loadings are statistically and theoretically significant • Educational/cultural possessions and parental education for cultural and economic capital • Parent-child cultural interactions for social and cultural capital

18. Discussion • But theoretical models seem not to hold between countries • SES measures (economic capital) are not comparable • This raises theoretical and methodological questions • Theoretical • Are other theories more appropriate? • Global or contextualized theories? • Methodological • What should be the groups of comparison? • Can ESEM manage comparisons among a large number of groups/countries?

19. Future Steps • Theoretical • Consider competing theories, e..g., rational action theory • Reproduction theories for developing countries and rational action theory for developed countries? • Methodological • MI • Identify problematic variables for MI • Countrywise comparison matrix • Evaluate MI within regions • What to do if no MI?

20. Questions? Thank you for your attention! :)