1 / 21

Exploring Individual Variability Using ACT-R

Explore sources of performance variability within and between subjects. Differentiate models and assess if averages represent individual behaviors. Investigate noise factors and their impact on adaptivity. Analyze an ACT-R model to study adaptivity variation and insights compared to Monte Carlo simulation.

beverleyk
Download Presentation

Exploring Individual Variability Using ACT-R

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Exploring Individual Variability Using ACT-R Christian Schunn George Mason University

  2. Understanding Variability in Performance • Within and between subjects variability are important sources of information (beyond average performance) • Differentiate models • Indicate whether average is representative of any individual's behavior (both within and between subjects)

  3. Sources of Variability: Naïve View • Random sampling with fixed probabilities • e.g., most mathematical models of memory • e.g., most mathematical models of choice

  4. ACT-R view • “Noise” (variations on Naïve) • Expected gain noise • Activation noise (encoding and transient) • Perceptual noise • Parametric variation • Global architecture differences • Ability (W, d) & motivational differences (G) • Experiential differences (expertise & luck) • e.g., q&r, a&b, activation, strength, aij, etc • Knowledge variation (expertise & luck) • Productions & Chunks

  5. An example: Are there individual differences in adaptivity? • On average, people select choices according to base-rates of success • e.g., Probability matching • On average, people adapt (or change) strategies when base-rates change • Reder 82, Siegler 87, Lovett & Anderson 96 • Do people systematically differ in how much or how fast they adapt? • Also, is average meaningful?

  6. A BST Experiment

  7. A BST Experiment

  8. A BST Experiment

  9. Experiment Details Block = 10 trials

  10. Aggregate performance

  11. A mathematical model of aggregate performance } r2=.93 (zero parameters)

  12. Variation in Adaptivity • Suggests more than just noise?

  13. ACT-R BST model • Adapted from Lovett (1998) • Same productions & parameter values • Force-over, force-under, then finish task, retry with other strategy if can't solve • Can ACT-R provide better fit and more insights than the Naïve Monte Carlo simulation?

  14. Fit to Data (default) r2=.77, RMS=.06 • Luck/experiential differences plus noise • Less variability than humans

  15. Sensitivity Analyses • What influences mean and variance in adaptivity? • EGS settings • Motivational levels • Learning • Prior experiences • Each model uses default settings and tweaks one feature

  16. Noise settings (EGS) .84, .08 .77, .06 .72, .09 • Even at better settings, variability is low

  17. Motivation settings (G) .84, .11 .77, .06 .67, .13 • At high G settings, variability goes down

  18. Learning decay (d) .75,.13 .77,.06 • Decay makes model too sensitive • And variability still too low

  19. Prior experiences .78, .25 .77, .06 • Apparently subjects have been playing BST previously? • Can get greater variability, but fit to mean becomes worse.

  20. Relationship to awareness data • Suggests more than just noise • ACT-R fits unaware best?

  21. Conclusions/Questions • ACT-R gives more persuasive exploration of chance variability • Variability and mean affected differently • EGS, G affect means but not variablity levels • For adaptivity only! (both affect block variability) • Amount of prior experiences affects both • Watch out for individual differences: • Evidence for parameter learning decay just a mixture of aware and unaware?

More Related