Topic 1: Cognitive Architecture The fixed parts of cognitive processes

1. Topic 1: Cognitive ArchitectureThe fixed parts of cognitive processes No matter what your favorite computational theory may be, it always assumes certain fixed properties of the system within which it functions

2. Among the architectural properties assumed by any computational theory: • The form of representations. Computational theories typically assume that representations consist of symbol structures. But how do they represent magnitudes? By numerals or analogs? • Available operations. The building blocks of processes that combine to form algorithms. • Constraints Information flow and on subprocesses (e.g., encapsulated modules?) • Fixed capacities determined by the innate structures of the mind/brain, as opposed to those that depend on their niche environment.

3. An even more fundamental reason why it is essential to get the cognitive architecture right is the critical role of "Cognitive Capacity" Because an organism may remain in its ecological or social niche, only a small fraction of its behavioral repertoire is ever actually observed. But an adequate explanatory theory must reveal an organism's structure, or its cognitive capacity. To do that it must account for the organism’s entire potential behavioral repertoire. That’s why “variance accounted for” is a poor measure of a theory’s explanatory value.

4. Observed regularity versus capacity:The difference between explanations that appeal to mental architecture and those that appeal to representations or tacit knowledge The parable of the found mysterious box: Walking through a field one day, a cognitive scientist comes upon a mysterious box. The box has a meter that records some aspect of its behavior. After many days of recording, the scientist finds some robust behavioral regularity of the box’s recording. What does it tell him about the nature of the box or about its intrinsic causal properties?

5. Observed Patterns of the Mystery Box (Double blip precedes single blip) (Single blip precedes double blip) time What does this behavior pattern tell us about the nature of the box?

6. An illustrative example: Mystery Box Careful study reveals that pattern #2 only occurs in this special context when it is preceded by pattern A What does this behavior pattern tell us about the nature of the box?

7. It tells us (very nearly) nothing about the nature of the system under study • Why? Because the observed behavior, although it is an objective true record, is but a small part of what the box is capable of. • The sample we observed is attributable to the environment – including what the box is used for – rather than to the fixed structure (the architecture) or thenatureof the box.

8. How can an objective record of behavior not tell you about the nature of a system? What more is there? • In this example, what the scientist found happens to be a box that is transmitting English messages in international Morse code. • Two short blips is a code for i, one blip is an e and a long-short-long-short sequence is a c. • Thus the regularity that the scientist found can be explained by the spelling rule in English: “i before e except after c”! • Nothing inside the box can explain that pattern because the box has a capacity not revealed by its usual behavior. It could transmit very many unobserved patterns because it has the capacity to do so.

9. The Moral: Regularities in behavior may be due to either: The inherent nature of the system or its structure or architecture. The content of what the system represents(what it “knows”).

10. Why it matters: A great many regular patterns of behavior reveal nothing more about human nature than that people do what follows rationally from what they believe. An example from language understanding The example of human conditioning

11. Another example where it matters:The study of mental imagery Application of the architecture vs knowledge distinction to understanding what goes on when we reason using mental images

12. Examples of behavior regularities attributable to tacit knowledge • Color mixing, conservation of volume • The effect of image size ? • Scanning mental images ?

13. What color would you see when the following two color filters overlap? ?

14. Where would the water go if you poured it over a full beaker of sugar? Is there conservation of volume in your image? If not, why not?

15. Studies of mental scanningDoes it show that images have metrical space? (Pylyshyn & Bannon. Described in Pylyshyn, 1981) • Conclusion: The image scanning effect is Cognitively Penetrable • i.e., it depends on goals and beliefs, or on Tacit Knowledge.

16. Do mental images have size?Imagine a very small mouse. Can you see its whiskers? Now imagine a huge mouse. Can you see its whiskers? Which is faster?

18. Part 2: Attention and Selection The next set of slides concern how information is filtered and encoded in order to alleviate the information overload faced by perceptual systems

19. There are many other more detailed properties of cognitive architecture that have been studied One of the most studied concerns the input systems: • How does the mind take in information (through perception)? • Since there is much more information in the perceived world than the brain is able to assimilate, how does it deal with this mismatch? • Among the mechanisms for dealing with speed mismatch (say in computers) are: • Buffering and fast shortcut ways of dealing with buffer contents (modularity of input systems) • Filtering (selective attention): Selection by location or what else? • Encoding in terms of more efficient codes (more efficient for our cognitive architecture)

20. Other findings relating to cognitive architecture: Input systems • Modularity of inputs. There is a large part of the first stages in vision that are encapsulated and insensitive to what the organism knows or believes (called Early Vision) • Selection of inputs to be encoded (Selective Attention) • A major area of study in Cognitive Science is the question of which properties are encoded in early vision (e.g., VAL) • Start with the observation that we see (in the sense of information getting through the eye) much more than we are able to deal with, so some filtering is necessary. • How and by what properties is information filtered? Not covered in this class

21. Broadbent’s Filter Theory Rehearsal loop Effectors Motor planner Senses Filter Limited Capacity Channel Very Short Term Store Store of conditional probabilities of past events (in LTM) Broadbent, D. E. (1958). Perception and Communication. London: Pergamon Press.

22.  Along what dimensions is human information processing capacity limited? • Channel capacity: Shannon-Hartley Theorem • This formula says that the capacity of a channel to transmit information depends entirely on such physical properties as its bandwidth in cycles/sec • But that does not work for human perception/memory. It seems that our capacity has to be measured in the number of coding units or “chunks” (George Miller’s classic paper “Magic Number 7 plus or minus 2”)

23. Example of the use of chunking • To recall a string of binary bits – e.g., 00101110101110110101001 • People can recall a string of about 8 binary integers. If they learn a binary encoding rule (000, 011, 102, 113) they can recall about 8 such chunks or 18 binary bits. If they learn a 3:1 chunking rule (called the Octal number system) they can recall a 24 bit string, etc

24. What does Selective Attention select? What is the basis for visual selection? • If attention is selection, what does visual attention select? • An obvious answer is places. We can select places by moving our eyesso our gaze lands on different places. • When places are selected, are they selected automatically? • Must we always move our eyes to change what we attend to? • Studies of Covert Attention-Movement: Posner (1980). • How does attention switch from one place to another? • Is it always the case that we attend to places? Can we attend to any other property? Can we select on the basis of color, depth, spatial frequency, affordances, or the property a painting has of having been painted by Da Vinci (A property to which Bernard Berenson was able to attend extremely well). cf Gibson

25. Covert movements of attention Example of an experiment using a cue-validity paradigm for showing that the locus of attention moves without eye movements and for estimating its speed.Posner, M. I. (1980). Orienting of Attention. Quarterly Journal of Experimental Psychology, 32, 3-25.

26. The object-based view of attention selection • There are good reasons for supposing that attention attaches itself to objectsrather than locations

27. We can select a shape even when it is intertwined among other similar shapes Are the green items the same? On a surprise test at the end, subjects were not able to recognize shapes that had been present but had not been attended – i.e., in this case they had not appeared in green.

28. If you do not attend to a visual pattern you may not see it! • Inattentional Blindness Mack, A., & Rock, I. (1998). Inattentional blindness. Cambridge, MA: MIT Press. • Change blindness

29. Another negative attention effect: Inattentional Blindness

30. Inattentional Blindness • The main task is to report which of two arms of the + is longer. • Subjects fixated their gaze at the intersection of the + lines • After many trials a critical trial occurs in which a small square appears. • On only this trial subjects were asked if anything different appeared • 25% of subjects failed to see the square when it was presented in the parafovea (2° from fixation). • But 65% failed to see it when it was at fixation! • When the background cross was made 10% as large, Inattentional Blindness increased from 25% to 66%. • It is not known whether this apparent “blindness” is due to concentration of attention on the primary task, or whether there is inhibition of outside regions.

31. What is attention is for? Treisman’sAttention as Glue Hypothesis • The purpose of visual attention is to bindproperties together in order to recognize objects

32. Read the vertical line of digits in the following brief display How are conjunctions of features detected? What color was the N? What color was the O? What letters were in red? Under these conditions Conjunction Errors are very frequent

33. Rapid visual search (Treisman) Find the following simple figure in the next slide:

35. This search is called a “popout” and does not require focused attention

36. “Objects” endure over time & space Several studies have shown that what counts as the same object endures over time and over changes in location – regardless of what properties these objects have and regardless of whether the properties change; This gives what we have been calling a “visual object” a real physical-object character and partly justifies our calling it an “object”.

37. Multiple Object Tracking • One of the clearest cases illustrating object-based attention is Multiple Object Tracking • Keeping track of individual visual objects requires a mechanism for individuating, selecting, accessing and maintaining the identity of individuals over time • These are the functions we have proposed are carried out by the mechanism of visual indexes (FINSTs) • We have been using a variety of methods for studying visual indexing, including subitizing, subset selection for search, and Multiple Object Tracking (MOT).

38. Multiple Object Tracking • In a typical experiment, 8 simple identical objects are presented on a screen and 4 of them are briefly distinguished in some visual manner – usually by flashing them on and off. • After these 4 “targets” have been briefly identified, all objects resume their identical appearance and move randomly. The subjects’ task is to keep track of which ones had earlier been designated as targets. • After a period of 5-10 seconds the motion stops and subjects must indicate, using a mouse, which objects were the targets. • People are very good at this task (80%-98% correct). The question is: How do they do it?

39. Multiple Object Tracking Experiments • Examples of Multiple Object Tracking displays used in our experiments can be viewed at FINST.com and also http://ruccs.rutgers.edu/faculty/pylyshyn/DemoPage.html