1 / 29

ASVAB: E Pluribus Unum?

Delve into the intricate relationship between cognition tests, general intelligence, and selection processes in the ASVAB. Explore how changing factor structures impact measurement invariance and test validity.

lmcglade
Download Presentation

ASVAB: E Pluribus Unum?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ASVAB: E Pluribus Unum? Martin J. Ippel, Ph.D. CogniMetrics Inc, San Antonio,TX StevenE. Watson, Ph.D. U.S. Navy Selection & Classification (CNO 132) Washington, DC 1

  2. The ASVAB is the principal instrument for selection and classification in the U.S. Armed Forces. Assumption: measurement invariance across full range of scores. Relevance: what is the “population of interest” of the ASVAB? Recent studies cast doubt on this assumption. 2

  3. Two related phenomena suggest a changing factor structure along the dimension of general intelligence (g): • The g factor gets smaller in high-g samples • Cognition tests have smaller loadings on “g” in high-g samples 3

  4. Spearman (1927) noticed already a decrease in the positive manifold of cognition variables at higher g levels. Spearman’s explanation: differentiation of intelligence 4

  5. The present study adheres to an alternative explanation: The phenomenon follows from the Pearson-Lawley selection rules. an underlying selection process changes the variance-covariance structure and the mean structure 5

  6. One phenomenon: • Decrease in positive manifold of cognition variables in • high-g samples differentiation of intelligence Two explanations: selection effects 6

  7. Consequences of: • differentiation: • selection effects: structure is changing underlying structure invariant 7

  8. Critical developments in psychometric theory: • Meredith (1964) showed that both the covariance structure and mean structure change if samples are selected based on one or more latent variables (e.g., the g factor). • Meredith (1965) developed procedures to derive the single best fitting (i.e., invariant) factor pattern derived from sets of factors obtained on populations differing on a latent variable. • Jöreskog (1971) formalized this viewpoint as an extension of the common factor model for a parent population to multiple groups based on one or more latent variables in the model. 8

  9. (df) Measurement Invariance: If we compare groups, or individuals of different groups, then the expected value of test scores of a person of a given level of ability should be independent of membership of these groups (Mellenbergh, 1989). In formule: f (Y | η, ν) = f (Y | η) y1ij= τ1i + λ1i ηij+ ε1ij fdepends on the measurement model of choice: 9

  10. invariant y1ij= τ1i+ λ1i ηij + ε1ij change 10

  11. Unequal intercepts 11

  12. ratings cluster 1 ratings cluster 2 η11 η 12 Unequal factor loadings 12

  13. a-select sample (n=1,000) Statistical Experiment: parent population (N = 48,222) hi-g sample (n=600) av-g sample (n=600) lo-g sample (n=600) 13

  14. a-select sample (n=1,000) determine factor structureand then sample Statistical Experiment: parent population (N = 48,222) hi-g sample (n=600) av-g sample (n=600) lo-g sample (n=600) 14

  15. Eigenvalues from a-select samples drawn from the parent population of Air Force recruits 15

  16. ASVAB tests and their measurement claims 16

  17. Model 1: A hierarchical model of “g” 17

  18. Model 2: A “g as first principal factor” model 18

  19. a-select sample (n=1,000) determine factor structureand sample Statistical Experiment: parent population (N = 48,222) hi-g sample (n=600) av-g sample (n=600) lo-g sample (n=600) 19

  20. Distributional properties of samples generated from the parent population based on a latent variable "g" 20

  21. ASVAB tests mean scores in samples with different levels of "g" 21

  22. Average correlation lower with lower variance 22

  23. The effects of selection based on the latent variable “g” on the variance of ASVAB tests 23

  24. Sequence of MCFA model fits and goodness of fit indices 24

  25. Non-standardized MCFA factor loadings. 25

  26. Standardized MCFA factor loadings 26

  27. should remain invariant y1ij= τ1i+ λ1i ηij + ε1ij change 27

  28. not invariant y1ij= τ1i+ λ1i ηij + ε1ij change 28

  29. Discussion: • ASVAB is measurement invariant in a limited sense: only factor loadings are invariant across different levels of “g”. (weak factorial invariance). • ASVAB seems to be measuring too many factors with too few tests. • more factors than eigenvalues larger than 1. • many tests have communalities < 0.60. • intercepts could not be constrained to be equal • (indicating: other factors influence test scores). 29

More Related