Clare Bucklin

1. Exploring the Psychometric and Neurological Relationship between Measures of Working Memory Capacity, General Fluid Intelligence Tests, and Potential Modality Interference Effects Clare Bucklin

2. The key structural difference between a Brown-Peterson (BP) working memory capacity (WMC) measure and a complex span (CS) one lies in how separate sets of processing (P) and storage (S) items are presented: -Interwoven presentation in CS -Blocked presentation in BP -Delayed span functions primarily as a baseline measure of STM span (WM tasks involve processing by definition)

3. Single (position-based) 2-back N-back tasks involve processing of the storage items in that participants are asked to determine whether a current stimulus matches another item presented “n” steps “back” in the sequence Dual (position- and content-based) 2-back

4. N-back vs. CS andBP • Thus, in n-back tasks, the storage items are used in the processing task, and they must be deleted after they are no longer relevant in the matching task • In CS and BP tasks, however, the storage item list must be retained in its entirety and in correct order for serial recall, and the processing task is distinct • The distinguishability of the processing items from the recall set can be manipulated, however • While many variations on the CS paradigm exist (operation span, digit span, reading span, etc.), it appears that few researchers, aside from Jarrold, Tam, Baddeley, and Harvey (2011), have attempted to modify the original BP paradigm (retaining one consonant trigram while counting back by 3s from a random three-digit number)

5. Complex Span vs. n-back on Validity • Since the 1990s, n-back tasks have been “the dominant measure in investigations of the neurological substrates of immediate memory” due to their face validityin that they involve S+P • However, “n-back has received little empirical validation as a WM measure,” whereas the complex span paradigm “has been extensively validated against other immediate memory measures, for example, by showing stronger relations to memory tasks requiring information manipulation than to those demanding mainly rehearsal” (Kane et al., 2007) • Authors did not specify validation against BP tasks

6. fMRI Evidence for a Link between WM and Fluid Intelligence • One of the key findings of an fMRI study of Vietnam veterans (n = 158) with focal lesions conducted by Barbey et al (2013) indicated thatthere was less overlap between the neural substrates of WM monitoring (rehearsal) tasks and gF: • Manipulating items in WM uniquely activated left-lateralized superior parietal/temporal areas, but shared right dLPFCactivation with gF tests (WAIS-III Perceptual Organization index) • Monitoring items (via single 1-, 2-, and 3-back tests) activated right middle temporal, right inferior parietal (only region that overlapped with gF measures) and right vLPFClocations • However, since complex span recall shows more correlation with manipulation tasks than n-back, these patterns of activation may have resulted from the specific combination of measures used (Kane et al., 2007)

7. Summary of Measures in Barbey fMRI study • General fluid intelligence (adaptive reasoning ability, as opposed to categorical knowledge encompassed by crystallized, or Gc) = WAIS-III Performance IQ Perceptual Organization Index • WM Monitoring (maintenance) = single n-back • WM Manipulation (transformation) = arithmetic problems and letter-number sequencing task • Verbal/numeric WM = digit span forward/backward • Spatial WM = spatial span forward/backward Authors were rather vague about the exact measure, but the Corsi block tapping test is often used

8. Barbey’s N-back based on Cohen et al. (1997) As one can observe, the maintenance and processing components are both letters, but, unlike in the Brown-Peterson paradigm, maintenance stimuli may be discarded (“updated”) after they are more than n steps back, plus there is no distinct set of processing stimuli or a distinct processing task.

9. Other WM tasks in Barbey fMRI study • The letter-number sequencing used to measure WM manipulation ability was similar (in terms of item type, not task) to the digit span forward/backward used to measure verbal/numeric WM: • Letter-number sequencing example: • X, 4, C, 1, 2, A  A, C, X, 1, 2, 4 • Digit span forward example: • 1, 5, 2, 7  “ “ • Backward: • 24, 2, 8, 15  15, 8, 2, 24 • Is it possible that this stimulus (as opposed to task) similarity contributed to the overlap between verbal/numeric WM and WM manipulation?

10. Overlap between WM task type and gF

11. Domain-specific WM overlap with gF

12. Verbal/numeric WM more strongly-correlated with manipulation, but this may have also resulted from too much maintenance-processing stimulus similarity between chosen measures (digit span for verbal, letter-number sequencing for manipulation, letters for all n-back conditions) Similar correlations between both memory modalities and gF, but (seemingly) more divergent for memory task type (manipulation vs. monitoring) Again, these correlations are based on single 1-, 2-, and 3-back conditions (no other paradigm tested)

13. Choosing Measures • Jarrold et al. assert that “the correlation between complex span tasks and academic abilities increases as the time available for maintenance activities” decreases • Less time allowed for monitoring  same effect as requiring more manipulation? • Possibly mediated by gF? • Surprisingly little research has been done on B-P tasks compared to complex span and n-back, which reportedly share as low as 2-5% of their variance • However, a more recent latent variable analysis by Schmiedek et al. (2009) found r = .96 (!) using reading, counting, and rotation span, as well as n-back and other updating tasks • Furthermore, on the RAPM gF test, verbal operation span performance accounted for 7% of variance, while single 2- and 3-back with n-1 lures collectively predicted 4% (Kane et al., 2007) • Schmiedek study found r = .78 for CS measures and RAPM and .84 for memory updating (including n-back) tests

14. Building on Another Previous Study • Jarroldet al. (2011) compared BP and CS performance • Storage item set: Monosyllabic concrete nouns and their pictorial equivalents, adapted from Snodgrass and Vanderwart (1980), to randomly-generate lists of maintenance (to-be-recalled) stimuli • Processing item set: A set of letter pairs chosen such that 1) half were rhyming and half not, and 2) half contained members that shared either a vertical or horizontal axis of symmetry • The verbal processing task consisted of making rhyming judgments, while the visuospatial one involved identifying whether a mutual axis of symmetry was present • BOTH PROCESSING TASKS USED THE SAME SET OF STIMULI • Controlled for baseline speed/accuracy on processing tasks and for baseline recall (delayed serial recall with no processing task)

15. Results of Jarrold et al. • Participants in both processing groups performed worse on the B-P in both the pictorial plus auditory and auditory only maintenance conditions, but this was especially true for the verbal processing group • Making rhyming judgments may have made maintenance via subvocal rehearsal more difficult; however, they did not use a pictorial-only condition • Concluded that “opportunities for attentional refreshment are greater on complex span” procedures, and posit that B-P task performance may be more strongly-correlated with measures of fluid intelligence (such as RAPM) than CS recall

16. Auditory + Pictorial Auditory only Notably, CS performance improved when pictorial presentation was removed for the nonverbal processing (mutual symmetry judgments) group, and performance got slightly worse for the verbal processing group when presentation was auditory-only BP performance declined by roughly the same number of items (~0.5) for both groups

17. Theoretical Implications for the Role of Attention • Importantly, the fact that BP performance was worse across both storage item presentation modalities and both processing tasks is congruous with the authors’ speculation that, “when all the memoranda are presented in a block, any refreshment or rehearsal is more likely to operate on the whole list and consequently is less likely to be successful.” • Would presenting the list as a single “chunk” (array) matter? • While the whole list must be recalled in CS as well, the gradual presentation of storage items may allow for greater attentional switching within WM, consolidation into LTM, or perhaps a combination

18. Cowan’s Embedded Process Model (1999) WM (STM + focus of attention) is regarded as an activated “subset” of LTM; furthermore, storage is not modality specific, and acknowledges possibility that rehearsal mechanisms other than speech-based ones exist (Chein et al., 2003)

19. Baddeley and Hitch’s Multicomponent Model (1974) In the MCM, by contrast, non-auditory information must go through the phonological store in a “loop,” whereas auditory input goes directly to it; the fact that Baddeley was a co-author on the Jarrold et al. study might explain why no visual-only condition was used Better overall recall for auditory info?

20. The Idea – WM Span Element • The procedure proposed on the following slides is quite similar to that used by Jarrold et al., but with a few key differences: • Brown-Peterson task only – plan to study n-back and CS individually later, then compare results of all three experiments • Pictorial-only or verbal (written)-only maintenance stimuli presentations – no auditory (spoken) or combined condition • Combined condition may have allowed for “dual encoding” • Debate over modality-specificity of maintenance processes • Removing auditory condition keeps experiment exclusive to visual system (participants will type responses in to corresponding serial positions) • Addition of original B-P consonant trigrams as maintenance stimuli set (goal of manipulating item-distractor similarity) • Both array and sequential presentations • Debate over whether focus of attention rotates between items or can hold multiple items simultaneously

21. BP Task: Array Presentation Condition Maintenance task Processing task Set sizes 4-7 (on screen for 750 ms times set size) Spatial (shared symmetry) OR Verbal (rhyming) Consonant Trigrams Snodgrass word Snodgrass pic. Hit “yes” key Hit “no” key OR OR Processing task intervals: 24 s for 4 maintenance items, 30 s for 5, 36 for 6, and 42 seconds for 7

22. Array Recall Processing task for 24 seconds (4 items) (program will track RTs to each letter pair) 3,000 ms (750x4) Program will track RTs Processing task for 30 seconds (5 items) ETC 3,750 ms (750x5) According to Gilchrist and Cowan (2011), activity in the intraparietal sulcus (IPS) “increased with a visual memory load and, unlike other brain areas, reached an asymptote in activity as the memory load approached the participant’s capacity limit.” – did not specify whether load was presented sequentially or in an array (another reason to limit design to vision for now) Incidentally, the IPS divides the superior and inferior parietal lobe – again, the right inferior parietal lobe was found to have overlapping activation during WAIS-III PIQ gF tests and WM monitoring tasks in Barbey et al. (2013)

23. Sequential Presentation/Recall ISI = 250 ms Each appears for 750 ms on screen fork cat tree ball NWB BMZ MQH CGW 1 2 3 4 5 Etc. for recall set sizes 4-7 (same processing tasks and processing intervals based on set size) For both presentation styles, participants will be able to type in their recalled items (vocalization was required in Jarrold et al. study)

24. Some Methodological Considerations • Because some of the pictorial one-syllable Snodgrass items could easily be identified by other names (e.g. “pistol” or “revolver” for “gun”), participants will be informed that all entries should be only one syllable; commonly-misspelled items also need to be controlled for (e.g. “ax” instead of “axe”) • Jarrold et al. only offered examples of rhyming letter pairs (verbal processing task) whose constituents also had (opposing) axes of symmetry (e.g. A and B) – does this mean I should exclusively use symmetrical letters even for the rhyming trials? • Also, “X” and “O” are the only two letters that have both horizontal and vertical symmetry, but the authors notably excluded these from their examples

25. Possible dual n-back variation

26. Fluid Intelligence Element • Administer RAPM • Tests of WM span and general fluid intelligence (gF) “are thought to capture variability in a crucial cognitive capacity that is broadly predictive of success, yet pinpointing the exact nature of this capacity is an area of ongoing controversy” (Mrazek et al., 2012) • Hall (1957) found r = .7 (n = 82) between the original WAIS Performance (gF) test and the original Raven’s (non-advanced) matrices; both had “similar age decline curves” • McLeod and Rubin (1962) found r = .68 (n = 81) • Unfortunately, I was not able to find more recent studies on this, but I wanted to ensure consistency with Barbey et al.

27. Why RAPM? • Barbey et al. used the Perceptual Organization index of the WAIS-III Performance IQ, which includes block design, picture completion, and matrix reasoning tasks • Raven’s Advanced Progressive Matrices are essentially a variation on the matrix reasoning subtest: • While the full RAPM test has 48 items (12 in Set I, 36 in Set 2), Chiesi et al. (2012) demonstrated that scores on Set I alone are sufficiently representative (n = 1,389) • Also, there appears to be no data regarding Brown-Peterson performance in relation to scores on either test WAIS RAPM

28. RAPM Tests: Pattern Recognition plus Mental Updating (Rotation and Transformation) In order to control for possible mutual practice effects between the visuospatial processing task (making judgments of whether mutual symmetry is present between two letters, which requires some mental rotation) and the RAPM questions related to rotation, counterbalancing will be required for both measures as well as all conditions within the BP-WMC measure -Jarrold et al. used the Latin squares design

29. Hypotheses • I am predicting a main effect of maintenance item-processing item similarity on recall performance • Since the consonant trigrams are clearly most similar to the capitalized letter pairs functioning as processing stimuli, an interference effect may occur (based on feature overlap) • I also predict an interaction between maintenance item-processing task similarity, in line with Jarrold et al.’s finding that word recall was worse after verbal processing • Conversely, picture and trigram recall after visuospatial processing may be worse • Finally, I hope to see a main effect of recall performance on RAPM scores, particularly for recall of non-verbal items after visuospatial processing • Higher resistance to same-domain interference from items mediated by gF?

30. I am also broadly predicting that array recall will be better due to the fact that items are presented simultaneously and for longer (750 ms x set size)

31. Four Notions of WM Recall Impairment • Oberauer (2009) outlined four commonly-tested hypotheses (none have become theories yet due to continued debate and conflicting evidence) on what causes recall decline in WM tasks while controlling for age and other factors manifestly affecting recall (e.g. lesions): • Distraction-of-attention: most related to the research of Barrouillet and colleagues (time-based resource sharing model, or TBRS); argues that WMC is directly related to cognitive load (CL), or “the proportion of time during which processing occupies attention” • CL = ta/T (time attn is occupied/total time)

32. TBRS: A Computational Version of EPM? • While acknowledging that “there is abundant evidence of between-domain interference between processing and storage,” Barrouillet et al. (2011) interpret this finding as evidence, in turn, for a domain-general attentional resource. • In contrast, interference-based accounts view forgetting of maintenance items as a result of “the nature of the items” in processing tasks, instead of the how long the processing tasks take to complete • Thus implies domain-specific argument for attention • Feature overwriting, response competition, superposition

33. Feature Overwriting and Hebb’s Rule • Hebb (1940s) is credited with the “cells that fire together, wire together” theory of neuron behavior • Oberauer and Kliegl (2006) expound on this notion in relation to WMC: “Units belonging to different items must fire out of sync. If each unit can fire only once per phase cycle, this implies that a feature unit can be bound to only one item at the same time.” • “items sharing a feature compete for the units representing that feature” • Contrasts with superposition, which contends that more recall distortion occurs when similar distractors become encoded with storage items. • Response competition = disadvantage for similar distractors due to larger candidate set in LTM

34. Quick Overview of C-SOB • The Contextual Serial Order in a Box (C-SOB) model, developed by Oberauer and colleagues (2012), assert what they view as a “new assumption” that “distractors create interference by being encoded into working memory.” • Furthermore, in a CS paradigm, “distractors are associated with the position of the immediately preceding item” – would associations be as strong in a BP paradigm, since presentation is not interwoven? • However, this group’s research has produced inconsistent findings as to the exact mechanism by which distractors create interference, but more recent work seems to favor novelty-gated encoding, which actually predicts that dissimilar distractors will be more strongly-encoded and thus more likely to interfere with recall – again, though, these findings have not yet been tested against a blocked/Brown-Peterson paradigm

35. Evidence for Enhanced Recall with Dissimilar Processing Stimuli (Oberauer & Kliegl, 2006) Performance on letter arithmetic operations (e.g. A + 3 = D) was found to improve when 4 phonologically-dissimilar letters were used for this type of WM manipulation task

36. More Evidence for Enhanced Recall with Dissimilar Processing Stimuli (Oberauer, Lewandowsky, et al.) Even though these results are from the same study advocating novelty-gated encoding, the authors stressed the important caveat that the more distractors are repeated (regardless of level of similarity), the less encoded they become

37. C-SOB, Cont’d. • Therefore, C-SOB “predicts no interference” between representations from distinct domains, assuming the processing stimuli are drawn from a finite, rotating (and thus non-novel) set • However, much uncertainty remains as to whether a “nominally visuospatial processing task involves only visual or spatial representations, and that a nominally verbal task involves only verbal representations.” • Asking participants to type their recall responses in after viewing pictures, then, might elucidate this matter somewhat (by comparing RTs from typing recalled items that were typed in the first place – words and trigrams)

38. Limitations • Some researchers have challenged the assumption that attention is required for maintenance in WM (e.g., Rerko & Oberauer, 2013) • Argue that “items are held in the region of direct access by virtue of being bound to their contexts, and focused information is held in a separate focus of attention.” • Context-binding may be more difficult/tenuous with blocked processing, however • Echoes Cowan’s Embedded Process Model (1999), a prominent alternative to the multicomponent model developed by Baddeley and Hitch (1974) • While Barrouilletet al. (2012) suggested that “encoding number words activates verbal representations susceptible to interference with letters to a greater extent than does encoding digits,” their results pointed to the duration, rather than type, of concurrent processing playing a bigger role in recall deterioration.

39. Results of the Cohen et al. (1997) study, which used the same n-back task as Barbey et al. (2013) The fact that the only overlapping area between load and temporal processing was in Broca’s area, which is responsible for language processing (n-back task included capital and lower-case letters) may speak to domain-specificity, but the effect of time alone impacted the superior temporal gyrus, also involved in lang. proc. However, fMRI methods and technology have advanced quite a bit since 16 years ago

40. Limitations, Cont’d. • While Jarroldet al. concluded that “verbal processing either blocks rehearsal of the memoranda or causes more forgetting via interference than does nonverbal processing because of the greater overlap between storage and processing materials in the former case,” there is research indicating that increasing item-distractor similarity may actually have a positive effect on recall in complex span (Rerko & Oberauer, 2013) • However, Rodriguez-Villagra et al. (2012) report that “interference by feature overwriting is larger among items within the same domain.” Thus, the letter processing stimuli might interfere with recall of trigrams and words more than for pictures

41. References • Barbey, A.K., Colom, R., Paul, E.J., & Grafman, J. (2013). Architecture of fluid intelligence and working memory revealed by lesion mapping. Brain Structure & Function (published online) • Barrouillet, P., De Paepe, A., & Langerock, N. (2012). Time causes forgetting from working memory. Psychonomic Bulletin Review, 19, 87-92. • Chein, J.M., Ravizza, S.M., & Fiez, J.A. (2003). Using neuroimaging to evaluate models of working memory and their implications for language processing. Journal of Neurolinguistics, 16, 315-39. • Chiesi, F., Ciancaleoni, M., Galli, S., & Primi, C. (2012). Using the Advanced Progressive Matrices (Set I) to Assess Fluid Ability in a Short Time Frame. Psychological Assessment, 24, 892-900. • Cohen, J.D., Peristein, W.M., Braver, T.S., Nystrom, L.E., Noll, D.C., Jonides, J., & Smith, E.E. (1997). Temporal dynamics of brain activation during a working memory task. Nature, 10, 604-7. • Gilchrist, A.L., & Cowan, N. (2011). Can the focus of attention accommodate multiple, separate items? Journal of Experimental Psychology: Learning, Memory, and Cognition, 37, 1484-1502. • Hall, J. (1957). Correlation of a modified form of Raven’s Advanced Progressive Matrices (1938) with the Wechsler Adult Intelligence Scale. Journal of Consulting Psychology, 21, 23-6. • Jarrold, C., Tam, H., Baddeley, A.D., & Harvey, C.E. (2011). How does processing affect storage in working memory tasks? Evidence for both domain-general and domain-specific effects. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37, 688-705. • Kane, M.J., Conway, A.R., Miura, T.K., & Colflesh, G.J. (2007). Working memory, attention control, and the n-back task: A question of construct validity. Journal of Experimental Psychology: Learning, Memory, and Cognition, 33, 615-22. • McLeod, H.N., & Rubin, J. (1962). Correlation between Raven Progressive Matrices and the WAIS. Journal of Consulting Psychology, 26, 190-1. • Oberauer, K. (2009). Interference between storage and processing in working memory: Feature overwriting, not similarity-based competition. Memory & Cognition, 37, 346-57. • Oberauer, K., & Kliegl, R. (2006). A formal model of capacity limits in working memory. Journal of Memory and Language, 55, 601-26. • Oberauer, K., Lewandowsky, S., Farrell, S., Jarrold, C., & Greaves, M. (2012) Modeling working memory: an interference model of complex span. Psychonomic Bulletin Review, 19, 779-819. • Rerko, L., & Oberauer, K. (2013). Focused, unfocused, and defocused information in working memory. Journal of Experimental Psychology: Learning, Memory, and Cognition (published online) • Rodriguez-Villagra, O.A., Gothe, K., Oberauer, K., & Kliegl, R. (2012). Working memory capacity in a go/no-go task: age differences in intereference, processing speed, and attentional control. Developmental Psychology, published online • Schmiedek, F., Hildebrandt, A., Lovden, M., Wilhelm, O., & Lindenberger, U. (2009). Complex span versus updating tasks of working memory: The gap is not that deep. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35, 1089-96.