Cognitive psychometrics and cognitive latent variable models

1. Cognitive psychometrics and cognitive latent variable models Joachim Vandekerckhove Department of Cognitive Sciences University of California, Irvine

2. Overview • The casting of covetous glances • Cognitive psychometrics and the diffusion model • Multilevel models • Explanatory modeling • Applications • Conclusions

3. Casting covetousglances

4. Casting covetous glances • Lee Cronbach’s 1957 Presidential Address • Experimental psychology • Systematic manipulation of variables • Focus on difference in group means • Correlational psychology • Measure latent psychological constructs • Focus on preexisting differences between groups • “Time to cross-breed!”

5. We are free at last to look up from our own bedazzling treasure, to cast properly covetous glances upon the scientific wealth of our neighbor discipline. Trading has already been resumed, with benefit to both parties.

6. Casting covetous glances • Lee Cronbach’s 1957 Presidential Address • Experimental psychology • Systematic manipulation of variables • Focus on difference in group means • Correlational psychology • Measure latent psychological constructs • Focus on preexisting differences between groups • “Time to cross-breed!” • Aptitude-treatment interaction research

7. Casting covetous glances • Present day • Experimental psychology • Systematic manipulation of variables • Psychometrics • Measure latent psychological constructs • Mathematical psychology • Modeling cognitive processes(Parameters with interesting interpretations) • Time to cross-breed! • Cognitive psychometrics

8. Cognitive psychometrics and the diffusion model

9. Cognitive psychometrics • Use cognitive models as measurement model • Try to explain differences • between trials, manipulations and persons • e.g. by regressing the parameters on covariates • Three “model building blocks” for explanatory modeling (De Boeck & Wilson, 2004) • Random effects • Manifest predictors • Latent predictors

10. Indexes p for persons, i for conditions Cognitive psychometrics • Most common measurement model: Gaussian • Normal linear model (linear regression, ANOVA): • But often not a realistic model • Entirely unsuited for, say, choice RTs

11. Diffusion model • Wiener diffusion model • Process model for choice RT • Predicts RT and binary choice simultaneously

12. Diffusion model • Wiener diffusion model • Process model for choice RT • Predicts RT and binary choice simultaneously • Principle: Accumulation of information • Interpretable parameters • Rate of information accumulation d • Information needed a • Initial biasb • Nondecision timet

13. τ d Evidence a z = ba 0.0 0.125 0.250 0.375 0.500 0.625 0.750 (For persons p, conditions i, and trials j.) time

14. (For persons p, conditions i, and trials j.) p(t) 0.0 0.125 0.250 0.375 0.500 0.625 0.750 time

15. Diffusion model • Wiener diffusion model • Process model for choice RT • Predicts RT and binary choice simultaneously • Principle: Accumulation of information • Interpretable parameters • Rate of information accumulation d • Information needed a • Initial biasb • Nondecision timet

16. Diffusion model • Classical methods: many associated problems • Substantive issues • Almost completely descriptive – differences over persons/trials/conditions cannot be accounted for in the model unless we model each cell separately • Technical issues • Parameter estimation / Model comparison • Difficult to combine information across participants • Problem if many participants with few data each • Problem if items are presented only once (e.g., words)

17. Multilevel modeling

18. Mixtures and mixing

19. τ d Evidence a z = ba qp 0.0 0.125 0.250 0.375 0.500 0.625 0.750 tpij Mixtures and mixing Across-person distribution Person-specific, across-trial distribution qp Prediction for trial (pij)

20. Mixtures and mixing This is a dominance parameter

21. gp li dpij Mixtures and mixing Item component Person component gp li (into Wiener process)

22. Mixtures and mixing • Addition of random effects • Allows for excess variability • Due to item differences • Due to person differences • Allows to build “levels of randomness” • Importantly, can be built on top of a diffusion model

23. Explanatory modeling

24. Explanatory modeling • Previous models were descriptive • Didn’t use covariates or variability over persons/items • Hierarchical models quantify variability • External factors can be used as predictors to explain the differences in parameter values (i.e., reduce unexplained variance) • Latent factors can be used to explain covariance between people/items/...

25. Explanatory modeling • Use basic “building blocks” for modeling • Random/Fixed effects • Person/Item side • Hierarchical/Crossed • Use covariates (continuous/categorical/binary, manifest/latent)

26. Explanatory modeling explaining variability in drift rate We translate the data into parameters, and explain variability through covariates

27. Implementation Vandekerckhove, Tuerlinckx, & Lee (2011). Psychological Methods.

28. Application 1The Leuven Natural Concepts data set

29. Leuven data • Features • Features of natural language concepts • Typicality, goodness, generation frequency, age of acquisition, word frequency, familiarity, imageability • Choice response times • Category verification task • E.g., “Is item dog a member of category mammals?” • Partial overlap in used categories • Birds, fish, insects, mammals, reptiles, musical instruments, tools, vehicles

30. Leuven data • A sneak peek at the data ... Item 51 Item 52 Item 53 Item 54 Item 55 Item 56 Item 57 Item 58 Item 59 Item 60 ... Person 11 0.9848 -1.4084 0.8513 0.7347 1.0622 0.8811 0.8563 0.9647 1.4821 0.7379 Person 12 0.7019 -0.7109 0.9987 0.7218 1.4516 0.7224 0.7065 0.8825 -0.7484 0.7842 Person 13 0.6202 -0.5820 0.6212 0.6829 0.5641 0.8666 0.5487 0.7380 0.6441 0.5972 Person 14 0.5771 1.7123 0.6004 0.9919 0.6308 0.7991 0.6567 0.7481 0.9260 0.8520 Person 15 0.5245 -0.6528 0.4329 0.8328 0.9855 0.5314 0.4700 0.4770 0.6685 -0.5828 Person 16 0.5999 -0.8530 0.7845 1.0349 0.7309 1.2436 0.6328 0.6482 1.1973 0.6922 Person 17 0.6108 1.6328 0.7291 0.8207 0.8655 1.3248 0.6365 0.6820 1.3392 1.0933 Person 18 0.5517 0.6534 0.5575 1.0180 0.5269 -1.9897 0.6728 0.7349 0.5815 0.6327 Person 19 0.7580 -1.0110 0.7540 0.7430 0.9048 0.7108 0.8420 0.6361 -0.8868 0.8387 Person 20 0.7072 -1.0976 0.6659 0.6209 0.7099 0.8687 0.7640 0.6860 0.9351 1.2465 Person 21 0.4902 1.3394 0.5202 0.5100 0.5158 0.5036 0.5924 0.5117 0.5062 0.6469 Person 22 0.6593 -1.1068 0.6861 0.6399 1.2217 0.7688 0.7624 0.6571 1.1753 1.0263 Person 23 0.6788 -0.5789 0.4865 0.4891 -0.5767 1.4254 0.5201 0.5019 0.9342 0.5797 Person 24 0.5178 -0.7130 0.5616 0.5622 0.6386 0.8462 0.6232 0.5124 0.5371 0.8966 Person 25 0.5833 -0.6059 0.5889 0.5903 1.0032 1.1688 0.5376 0.5652 -0.8289 1.3995 Person 26 0.6885 0.8769 0.6630 0.6793 1.0409 0.6269 0.6604 0.7518 0.6824 0.9248 Person 27 0.7425 0.7650 0.7280 0.8343 1.4173 0.7811 0.6101 0.6166 0.9403 0.6284 Person 28 0.5630 -1.5076 0.6440 0.5725 0.7045 0.6869 0.6113 0.5552 1.0313 0.6628 Person 29 0.6716 -0.8894 0.6406 0.8293 0.6546 -1.0920 0.7749 0.6582 0.9315 1.2917 Person 30 0.6673 1.0799 0.7001 0.8787 0.7200 -1.0442 0.6390 0.7814 0.9698 0.6540 ...

31. Leuven data • A sneak peek at the data • Importantly, each stimulus only shown to each participant once • Distributional analysis per cell is impossible • Cannot collapse over items • Shouldn’t collapse over persons hierarchical analysis • We want to disentangle person and item effects on drift rate • Crossed random effects design

32. Indexes p for persons, i for words Trial’s mean drift rate is a sum of person aptitude and item easiness Leuven data Item easiness distribution may differ according to whether the item is a target or a distractor

33. Leuven data: Model Check Mean RT item Accuracy item

34. Leuven data • Some results Estimand Post.mean Estimand Post.mean Pop. distr. of item easiness item easiness person aptitude

35. Leuven data

36. Drift rate regression

38. Nondecision time regression

40. Leuven data • Drift rate results • Typicalitya ‘significant’ predictor in most categories • No other predictor shows consistent pattern • Nondecision time • Word Length a ‘significant’ predictor in most categories • No other predictor shows consistent pattern

41. Conclusions • Leuven data • Variance in person aptitude small (≈ 0.04) relative to variance in item easiness (≈ 0.11) • Item easiness correlates with typicality • Nondecision time correlates with word length Vandekerckhove, Verheyen, & Tuerlinckx (2010). ActaPsychologica.

42. Application 2Effects of valence in a proactive interference task

43. Factor analysis • Partials out underlying skills from one another and from task-specific abilities • Marginal distribution is often Gaussian φ1 φ2 Skills Task-specific abilities b1 b2 b3 b4 b5 Raw data points, normality assumed

44. Cognitive factor analysis • In many cognitive studies, the “score” on a lab task should be the participant’s rate of information processing (rather than just their accuracy) φ1 φ2 Skills Task-specific parameters b1 b2 b3 b4 b5 Performance in experimental tasks

45. Cognitive factor analysis φ1 φ2 Skills Model predicted behavior Task-specific parameters b1 b2 b3 b4 b5

46. Identification constraints

47. Proactive interference task A B C D E? interference E F G H G? I J K L F?

48. Adding more covariates • Battery of personality covariates • CESD, SWL, RRS… • Processing deficits in dysphoric participants expected • Latent factor scores • Can project covariates into factor space and interpret factors and loadings

49. Model selection result • “Winning” model had F = 6 factors: • Intercept, detection (nonrecent), detection (recent), PI (neg), PI (pos), PI (neu)

50. Covariate loadings • Person covariates mapped into factor space