Perceptual Learning, Roving and the Unsupervised Bias

1. Perceptual Learning, Roving and the Unsupervised Bias By Aaron Clarke, Henning Sprekeler, Wolfram Gerstner and Michael Herzog Brain Mind Institute ÉcolePolytechniqueFédérale De Lausanne Switzerland

2. Talk Outline • Perceptual Learning & Roving • The Unsupervised Bias • Critical Experiment

3. Perceptual Learning

4. Perceptual Learning 4.5 4 3.5 3 2.5 d' 2 1.5 1 0.5 0 0 5 10 15 20 25 30 35 40 45 50 Block Number

5. Talk Outline • Perceptual Learning & Roving • The Unsupervised Bias • Critical Experiment

6. Roving Learning Task 1 1200” 1200” 4.5 4 3.5 3 2.5 d' 2 1.5 1 0.5 0 0 5 10 15 20 25 30 35 40 45 50 Block Number

7. Roving Learning Task 1 Learning Task 2 1200” 1800” 1200” 1800”

13. Roving Non-Roved Roved 4 4 1200" 1800" 3.5 3.5 3 3 2.5 2.5 d' d' 2 2 1.5 1.5 1 1 0.5 0.5 20 25 0 5 10 15 20 25 0 5 10 15 Block Number Block Number Adapted from Tartaglia, Bamert, Mast & H. Herzog (2009)

14. Hypotheses • Roving may disrupt memory-trace-buildup for the roved stimuli (Yu et al., 2004). • Roving may diminish the stimuli’s predictability (Adini et al., 2004). • Roving may prevent the participants from conceptually tagging each stimulus type in order to switch their attention to the appropriate perceptual template (Zhang et al., 2008).

15. Roving Learning Task 1 Learning Task 2 1200” 1800” 1200” 1800”

16. Hypotheses • Roving may disrupt memory-trace-buildup for the roved stimuli (Yu et al., 2004). • Roving may diminish the stimuli’s predictability (Adini et al., 2004). • Roving may prevent the participants from conceptually tagging each stimulus type in order to switch their attention to the appropriate perceptual template (Zhang et al., 2008).

17. Talk Outline • Perceptual Learning & Roving • The Unsupervised Bias • Critical Experiment

18. Talk Outline • Perceptual Learning & Roving • The Unsupervised Bias • Critical Experiment

19. Model Predictions Unsupervised Supervised Reward-Based Δwij = prei ×postj Δwij = prei ×eij Δwij = Cov(R,wij) + ‹R› ‹wij› Desired Output Desired Output Output Output Error Error j i Reward Input Input Input • No feedback • Trial by trial feedback • Error feedback • Teacher signal • Feedback after many trials • Error feedback • Teacher signal

20. Model Predictions Unsupervised Supervised Reward-Based Δwij = prei ×postj Δwij = prei ×eij Δwij = Cov(R,wij) + ‹R› ‹wij› Desired Output Desired Output Output Output Feedback improves performance. Learning is possible without feedback Error Error j i Reward Input Input • No feedback • Trial by trial feedback • Error feedback • Teacher signal • Feedback after many trials • Error feedback • Teacher signal Herzog & Fahle (1998)

21. Reward-Based Learning Δwij = Cov(R,wij) + ‹R› ‹wij› Reward & current activations Averages of past trials weight change Covariation between reward weight change Average reward

22. Reward-Based Learning Δwij = Cov(R,wij) + ‹R› ‹wij› Reward & current activations Averages of past trials weight change = 0 Covariation between reward weight change Average reward

23. Reward-Based Learning Δwij = Cov(R1+R2,wij) + ‹R1+R2› ‹wij› Reward & current activations Averages of past trials weight change Covariation between reward weight change Average reward • Learning is impossible with two stimuli.

24. Roving Non-Roved Roved 4 4 1200" 1800" 3.5 3.5 3 3 2.5 2.5 d' d' 2 2 1.5 1.5 1 1 0.5 0.5 20 25 0 5 10 15 20 25 0 5 10 15 Block Number Block Number Adapted from Tartaglia, Bamert, Mast & H. Herzog (2009)

25. Talk Outline • Perceptual Learning & Roving • The Unsupervised Bias • Critical Experiment

26. Hypothesis • Roving impairs perceptual learning when the average reward for the two learned stimuli differs significantly. • This kind of situation occurs when the two roved tasks differ in their difficulty levels.

27. Roving Learning Task 1 Learning Task 2 1200” 1200” 1800” 1800”

28. Results H0: Mean Hard Slopes = 0: t(7) = -1.115, p = 0.151 4.5 1200” Easy Hard 4 3.5 3 2.5 d' H0: Mean Easy Slopes = 0: t(7) = -0.222, p = 0.415 2 1.5 1800” 1 0.5 0 0 5 10 15 20 Block Number

29. Results 4.5 4.5 Easy H0: Mean Non-Roved Slopes = 0: t(7) = 2.144, p = 0.035 Hard 4 4 3.5 3.5 3 3 2.5 2.5 d' d' 2 2 1.5 1.5 1 1 0.5 0.5 0 0 0 5 10 15 20 0 5 10 15 20 Block Number Block Number

30. Summary • There are three types of learning models: supervised, unsupervised and reward-based. • Only reward-based learning withstands empirical falsification, and it suffers from the unsupervised bias. • When roving two tasks, easy and hard, learning fails, as can be shown mathematically. And that is why roving occurs empirically. • A strange prediction from this is that roving a hard and a very easy task should deteriorate performance. Roving two hard tasks might make learning easier than roving a hard and an easy task, and this has actually been shown in other studies.

31. Thank for your attention.

32. When is Learning During Roving Successful? Vs. Vs. Vs. 150 ms 500 ms

33. Experiment • Used two stimuli: 1800” and 1200”. • Measured pre-training thresholds for both stimuli in isolation. • Trained subjects with fixed offsets (easy = 1.5 × pre-training threshold, hard = 0.9 × pre-training threshold). • In 20 blocks of 80 trials. • Roved stimuli. Easy 1200” Easy 1800” Hard 1200”

34. Other Hypotheses • Roving may interact with the participants’ initial performance levels where worse initial performers learn more than high initial performers. • Roving might cause low-level interference between stimulus types (Tartaglia et al., 2009; Zhaoping, Herzog, & Dayan, 2003).