Chapter 2: Modeling mental imagery

1. Chapter 2:Modeling mental imagery

2. The ingredients Encountered some of the basic ideas feeding into cognitive science • move away from associationist models of learning and behavior • information theory as a tool for exploring the nature and limits of cognitive abilities • development of “boxological” accounts of how cognitive tasks can be performed • theory of computation as a model for information-processing Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

3. Putting them together: 3 case studies • Terry Winograd and SHRDLU [TODAY] • The imagery debate [MONDAY] • Marr’s theory of vision [WEDNESDAY] Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

4. Earlier themes The nature of mental representation – Miller and chunking  information-processing depends on how information is coded – Winograd and procedural semantics  representation of “knowledge” in terms of algorithmic routines Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

5. Common assumptions about information Information is amodal • Miller’s suggestion that the sensory systems all have the same channel capacity Information is coded in a digital/propositional format • based on the formal languages used to program computers Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

6. Digital information-coding Information is coded in a format that has the basic properties of a language • Basic constituents are individual symbols • Compositionality – complex structures are built up from individual symbols according to formation rules • Arbitrary connections between symbolic structures and what they represent Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

7. Digital information-processing The model for thinking about digital information-processing are formal languages (e.g. logical languages and computer programming languages) Model information-processing on, e.g. • proofs in logical languages • implementation of instructions in a production system Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

8. Imagistic information-coding Non-symbolic: images are not built up from basic elements Not compositional • The parts of images cannot reoccur in other images • No rules for building up images from their parts Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

9. Representation in images Representation depends upon systematic correlation between properties of representation and properties of what it represents pictorial depiction depends upon resemblance can be schematic resemblance, as in a map Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

10. Tricky issues Imagistic representations can exploit symbols (e.g. maps) • need to distinguish between the representation and the labeling of the representation Imagistic representations ≠ analog representations • a representation is analog just if it permits continuous variation • there are examples of analog representartions that are not imagistic and imagistic representations that are not analog Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

11. Imagistic information-processing The real issue comes with how information is extracted from imagistic representations • scanning images • manipulating images (e.g. rotation) Certain types of information are much easier to extract from images than from digital representations Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

12. The issues for cognitive science Is information always encoded in a digital format - or are there cases of imagistically encoded information? How can we explore this experimentally? • By looking at how subjects carry out information-processing tasks involving images • Seeing whether their behavior provides indirect evidence that they are scanning/manipulating images Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

13. Brooks 1968 F • Form a memory image of a capital F • Trace around the image, starting at the bottom left corner and working clockwise • Indicate for each corner whether it is on a top edge of the figure • Performance is impaired when responses are made visually (i.e. by pointing to the word ‘Yes’), rather than by saying ‘yes’ Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

14. Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

15. Cooper 1975 Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

16. Scanning mental images Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

17. The strong interpretation Subjects perform the task by rotating/scanning mental images in their “mind’s eye” The process of mental rotation/scanning has is structurally similar to physical processes of rotation/scanning Seems to match evidence from introspection Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

18. Problems with the strong interpretation Dennett’s “Cartesian theatre” Who or what is doing the scanning/rotating? Where is the image projected? Threat of regress if we take the metaphor of the “mind’s eye” literally Not clear how these mental images relate to “phenomenal images” Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

19. Kosslyn’s theory • Develops metaphor of images as spatial displays on cathode ray tube • Mental images are temporarily generated from propositionally encoded information in long-term memory • Mental images “projected” onto visual buffer (which is where perceptual representations also appear) Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

20. Solving a problem Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

21. Ambiguity Personal-level phenomena phenomenal images conscious experience of the world accessible to introspection (not always reliable) Subpersonal information-processing  explains our personal-level phenomena and abilities Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010