What finger movements reveal about cognitive processing during Braille reading

1. Barry Hughes Department of Psychology Research Centre for Cognitive Neuroscience University of Auckland Terri Hedgpeth Disability Resources Center Arizona State University What finger movements reveal about cognitive processing during Braille reading

2. General hypotheses Finger movements during Braille reading are useful as a window onto the otherwise invisible processes that underlie Braille reading. Analyses of finger movements will contribute to • the development of formal models of perceptual-motor coordination during reading • linking functional cognitive models to distribution patterns of neural activity • insights into the learning and teaching of braille

4. Finger position by time Potential factors that could influence the left-right velocity of reading finger(s) Haptic (skin-surface) interactions Motor control factors Linguistic processing demands Finger velocity by time

5. General Methods Participants (N = 23) Age ranged from 22 y to 75 y; mean: 45.8 (+ 17.6) y. Mean experience as fluent readers : 35.5 (+ 18.1) y. Under normal circumstances, six participants used a single finger to read; 16 read with the index finger of each hand and one read with the index and middle fingers of each hand. For 19 readers the right indexfinger was designated dominant; for four readers the left index was designated dominant.

6. Movement Recording and Analysis The grip pen for a high resolution digitizing tablet (Wacom Intuos) was fitted to a light-weight finger attachment on the reader’s dominant reading finger.

7. Study 1: Scanning vs reading Braille Experiment in two phases, presented in a counterbalanced order 1. A phase in which readers were required to read but merely to move the finger across as (“scan as smoothly as possible”) meaningless strings of Braille cells. 2. A phase in which readers were required to accurately read sentences (either aloud or to repeat verbatim). All sentences, whether scanned or read, were within a single line of text. All sentences to be read were rendered in Grade 2 (contracted) Braille.

8. In all cases the sentences took a template form: article [adjective] noun verb article adjective noun. Sentences were created by crossing two factors: Word [noun, verb, adjective] frequency (high or low) Sentence meaning (‘meaning’ or ‘nonsense’) Four sentences of each type were read twice in a random order. 1. high frequency words, meaning A father [213] imagines [71] his quiet [74] year [678]. ,A "F imagines 8 quiet year4 2. low frequency words, meaning 3. high frequency words, nonsense 4. low frequency words, nonsense A blurb [1] calculates [1] the bottled [2] walnut [3]. ,A blurb calculates ! bottl$ walnut4

9. Control Scanning

10. Reading

11. For all dependent variables, ANOVA revealed effects of word frequency (H v. L) re-reading (R1 v. R2) and their interaction reading mode (silent v. oral) sentence meaning (M v. N) had no significant effect on any dependent variable.

12. There appears to be a strong exponential and inverse relationship between the mean velocity with which a sentence is read and the number of inflections in the velocity trace. This strengthens the claim that the smoothness of the velocity trace is largely determined by the finger’s mean velocity, rather than by language processing directly.

13. Study 2 If word frequency has an effect on mean velocity and smoothness and reversals, is this because (a) the words are more frequently occurring? or (b) the orthographic features of the words are more familiar? Participants read sentences that had similar openings but concluded with key words, including an adjective and a noun. These key words involved combinations of word frequency (high or low), and orthogonal familiarity (high or low)* * Statistics from E-lexicon Project http://elexicon.wustl.edu/ See also Balota et al (2007). Behavior Research Methods, 39 (3), 445-459.

14. Words High-Orthography High HH,%e sad %e _h a r1sona# morn+4She said she had a reasonable morning.Words High-Orthography Low HL,%e sad %e _h unusual ?"\s4She said she had unusual thoughts.Words Low-Orthography High LH,%e sad %e _h mangl$ 3te/s4She said she had mangled contests.Words Low-Orthography LowLL,%e sad %e _h newf.d idiosyncrasies4She said she had newfound idiosyncrasies.

15. Words High-Orthography High HH ,Ty 3sid}$ x a will# 3nec;n4 They considered it a willing connection. Words High-Orthography Low HL,Ty 3sid}$ x a typical faculty4 They considered it a typical faculty. Words Low-Orthography High LH,Ty 3sid}$ x a 3d5s$ repres.n4 They considered it a condensed repression. Words Low-Orthography Low LL ,Ty 3sid}$ x a noxi\s typhoid4 They considered it a noxious typhoid.

16. As expected, there was little difference in kinematic variables during opening segments • In the key word segments, we find • an increase in velocity (especially for high frequency words); • an increase in acceleration zero-crossings (especially for low frequency words); • an increase in reversals (especially for low frequency words) • but • no effects of orthographic familiarity

17. Study 3: On reversals Reversals are a distinct category of movement intermittencies in Braille reading. What causes reversals in Braille reading? • are they regressions to recover from explicit errors of comprehension? • are they movements that serve to synchronise the finger position with the linguistic processing (the ‘finger-mind span’)? • are they the second, return part of fast look ahead movements?

18. We asked readers to silently read sentences to completion, at which point a symbol would indicate which one of the sentence’s two nouns should be reversed to as fast and as accurately as possible. The sentences came in three forms: normal: grammatical and coherent The sickly maid took the pill. The motorists smashed the glowing headlights. reversed: grammatical but not semantically coherent The tasty eggplants consumed the goat. The aroma smelt the sneaky predators. scrambled: neither grammatical nor coherent Frail held snapshots the the historian. Costly teens the the admired yacht. The target noun could be: located in the noun phrase or the verb phrase of a normal sentence) short or long in length (4 cells or 8 cells in Grade 2 Braille)

19. Normal sentences: grammatical and coherent

20. Reversed sentences: grammatical but not coherent

21. Scrambled sentences: Not grammatical, not coherent

22. Summary • Kinematic analyses reveal important details of the reading finger • that “smooth” Braille reading is almost always an illusion; • that processing language more efficiently results in an increase in velocity and only indirectly in smoother movements; • that some factors (such as letter combination frequency, mode of reading and sentence meaning) have no measurable effect on reading velocities; • that Braille readers reverse movement direction more often than print readers –and perhaps for different reasons; • that reversals are fast but not ballistic movements, with contact maintained with the reading surface.

23. Research has been supported by grants from Bilateral Research Assistance Programme of the Royal Society of New Zealand Faculty of Science Research Development Fund Collaborators Arend Van Gemmert, Louisiana State University George Stelmach, Arizona State University Hans-Leo Teulings, Neuroscript LLC AshwinMathur, University of Auckland Phillipa Turner, University of Auckland Mayuri Patel, University of Auckland