NEURO VISION

1. NEURO VISION

2. What is so Great about it? • Deceptively simple anatomical appearance. • Incredibly complicated structure. • Developed through millennia of evolution. • Hard-wired to prefer certain objects right at birth. • Invariant to position, scale and rotation of the object. • “Tuned” to quickly recognize objects.

3. IS HUMANVISION PERFECT ?

4. Tricks of the brain PENROSE’S TRIANGLE

5. I’d love to see someone try to get to the top PENROSE’S STAIRS

8. Why is it so? • Our vision system neural network has been tuned to perform recognition, processing and classification of phenomena that was vital to our survival and progress. In this context, every species has a different vision system. • Hence, we are not very good in dealing with artificially generated images, as these phenomena rarely occurred in nature during our evolution. • However, we are the best for natural images!

9. How human visual perception works.

10. How human visual perception works • Perceptions of static scenes are inadequate to describe motion • Gibson’s theory of affordances • Vision evolved in organisms embedded in a dynamically changing environment • What is important to an organism is a collection of processes, not a single unique one • These processes are at different levels of abstraction. E.g. We see waves on a shore, and also the innumerable molecules in it moving

11. How human visual perception works(contd.) • “Seeing involves multi-level process simulations in partial registration using different ontologies, with rich (but changing) structural relations between levels” • Use of structures of various sorts • Agglomeration/grouping: Structures of different sizes at same level of abstraction • Interpretation: Structures at different levels of abstraction-mapping to a new ontology • Fragments recognized in parallel, assembled into larger wholes-may trigger higher level fragments, or redirect processing at lower levels to resolve ambiguities, etc.

12. Functions of vision • Segment the image (or scene) and recognize the objects distinguished • Compute distance to contact in every direction • Provide feedback and triggers for action • Provide a low-level summary of the 2-D and/or 3-D features of the image, leaving it to the central non-visual processes to draw conclusions • Is something left out?

13. Visual/spatial reasoning • Our ability to use diagrams and visual images to reason about very abstract mathematical problems, like thinking about the complexity of a search strategy • “Seeing” that 7+5=12 by a rearrangement of dots • “Seeing” that angles of a triangle add up to a straight line • Visualize infinitely thin and long lines of Euclidean geometry • Many more examples

14. Visual/spatial reasoning(contd.) • Uses of spatial reasoning: Knowing where to search for an object thrown over a wall, assembling toy crane from a toy set, uses of spatial concepts(notion of search space) in programming design • Reasoning using a grasp of spatial structures requires at least: the ability to see various structures involved in the proof, the possibilities for variatins(rearrangements) in them, the invariant structures during the rearrangements, etc. • In contrast, a reasoning system like logic is completely discrete and all syntactic composition involves function appllication • Specification of the requirements for visual reasoning is very vague, and would not be easy to mechanize

15. Visual perception involves much more.. • Visual perception involves “affordances” • Affordances are the possibilities for, and constraints on action and change in a situation. Seeing the possibility of things that do not exist, but might exist • Example: A person perceiving a chair can immediately see the possibility of sitting on it, that is, the chair "affords" sitting

16. Visual perception involves much more..(contd.) • POPEYE(1970’s): The Popeye project investigated how it is possible for humans to see structure in very cluttered scenes, where structure exists at different levels of abstraction-it showed that we recognize fll words before individual alphabets • Consider looking at a smiling or a sad face. Does it involve only perceiving the structure of the pattern? We are able to perceive mental states of happiness or sadness

17. Visual perception involves much more..(contd.) • What may appear to be only one task, might consist of many different tasks in different contexts, e.g. estimating the length of a plank to fit across a ditch • When a number of images are speedily flashed before the eyes in order, the speed with which people can see at least roughly, what sort of scene is depicted by each image, implies that our visual mechanisms are capable of finding low level features, using them to cue in features of the images at various levels of size and abstraction, arriving at percepts involving known types of objects, within 1 or 2 seconds • High level precisions are made in less than 1/2 a second

18. Artificial Vision systems-what they aim at • recognize objects or people in static images without acquiring or reasoning about the information using 3-D structure • track moving objects represented in simple shapes(points or blobs) often using 2-D representations • explore an environment building a 2-D map of walls, doors, etc. without a possible human understanding of the maps • control a moving robot, regarded as a moving object • obtain some 3-D information about the environment, only to generate new images

19. What vision systems cannot do • After Freddy, the Edinburgh robot built in 1973, there was a need to move from 2-D to 3-D. Failure due to limitations of computational power, and difficulty of choosing a representation • Consider a cup on a table. Humans can "see" the orientation required for the grasping object at different grasping locations-visual systems cannot. Alignment of grasped surfaces with grasping ones is important • Affordances in the object being grasped, if it has sharp corners, some part of it is more fragile than others-requires a grasp of counterfactual conditionals involving processes that do not actually exist

20. Why the limitation? • To develop a human-like visual system that will do what a small child/many animals do-need for an adequate analysis of the requirements for such a system • The requirements might seem much simpler than they actually are, if they are not studied in sufficient depth • The failure to achieve set goals is not a fault of the choice of domains, or the representation-it is a problem of overoptimistic predictions

21. Where is the complexity? • Different levels of perception needed. High level of precision to lift a hair with a pair of tweezers, much lower precision to see something is not graspable • Perception can involve multi-strand relationships requiring much richer forms of representation that just a logical form • Multiple levels of abstraction, affordances, causation-all is needed • Many more subtleties..

22. Visual Pathway Hierarchical Neural Network Architecture

23. Contents • Brain Mechanism of Vision • Hubel‘s and Wiesel's hierarchy model

24. Cerebral Cortex • Evolution of cerebral cortex is one of the great success in the history of living beings. • Insights of cortical organization: • Division into different regions having different functionalities. e. g. , Visual, auditory, somatic sensory, speech and motor regions

25. Visual Pathway • Retina to the Visual Cortex

26. Hubel’s and Wiesel’s Model • Hierarchical model of cortical cells . The cortical cells are divide into various types • Type IV • Simple cells • Complex cells

27. Hubel’s and Wiesel’s Model Type IV • Cells have circular symmetry. • The receptive field of the cell is divided into • on Center. (Excitatory Center and Inhibitory Surrounding) • off Center. (Inhibitory Center and Excitatory Surrounding)

28. Hubel’s and Wiesel’s Model Simple Cells Respond to an optimally oriented line in a narrowly defined location. Achieved by requiring the centers of layer Iv cells that lie along the line.

29. Hubel’s and Wiesel’s Model • Complex cells the main feature of complex cell • They are less particular about the location, Concerned mainly on orientation. • Aquired is from a number of simple cells • Detects motion(direction specific).

30. Hubel’s and Wiesel’s Model

31. Biological Visual Systems as Guides Modelling attempts to imitate primate vision systems

32. Extended Hubel-Weisel • Hubel-Weisel hierarchical models have been extended to obtain a fine balance between selectivity and invariance. • Simple and complex cells are interleaved at different levels of the inferotemporal (IT) lobes. • Max-like pooling mechanisms have been suggested at certain levels as opposed to a weighted sum of afferents to boost invariancy in scale, position and rotation.

34. Feedforward Architecture • The S cells (simple cells) in the previous figure passed on information to the C cells (complex cells) by a bell-tuned weighted sum or a max-like operation. • These cells were further arranged in a higher feature-level hierarchy. • Some cells bypass a level in propagating information. • This model only considers the feedforward architecture model for the primary visual cortex, V4 and the posterior IT lobe, and a top-level supervised learning mode (coloured regions). [Serre et al. 2007]

35. Feedforward Architecture • Primates have a very advanced level of attention modulation (fixation) which is a feedback propagation from the IT lobes to the primary visual cortex and lower levels. • This mechanism allows to shift attention from one part of the image to another. • However, crude object recognition is done in a very small duration after stimulus which indicates use of only the feedforward architecture for rapid categorization. • Such a model was attempted at the McGovern Institute for Brain Research at MIT with some simplifications. • The input consisted of 4 different orientations and several scales, densely covering the gray-value input image of 7ºx 7º

36. Results • The model was evaluated against human responses for input stimulus of 20ms followed by varying inter-stimulus interval. • No single model parameter was adjusted to fit the human data. All unsupervised parts were fixed and constant throughout all the runs. • The supervised mode was tuned differently in different runs using different test images. Humans were also shown these test images. • An evaluation across all such runs for the identification of animal objects was done for both humans and animals. The results were compared.

37. Results • Various categories of images in different clutter, scale, position, rotation were given. • Maximum similarity was found for ISI until 80ms.

39. Conclusions • Biologically inspired computation models have shown very promising results. They are versatile and fast learners. Why not learn from nature’s best? • Advances in neuroscience are picking up, allowing us greater understanding. Also, simulations of hypothetical models will help us validate neuroscience findings.

40. References • Talks by Aaron Sloman, Univ of Birmingham, UK 2005 - 2007http://www.cs.bham.ac.uk/~axs/invited-talks.html • http://www.lifesci.sussex.ac.uk/home/George_Mather/Linked%20Pages/Physiol/Cortex.htmlLast accessed - 13 April 2008 • Brain Mechanism of Vision, David H. Hubel and Torsten N. WieselScientific American, September 1979 • How We See What See - V. Demidov, Mir Publishers, 1986 • A feedforward architecture accounts for rapid categorizationSerre et al., PNAS, 2007 • Hierarchical Models of Object Recognition in Cortex Poggio et al., Nature America, 1999 • http://www.thebrain.mcgill.caLast accessed - 13 April 2008