200 likes | 381 Views
Cognitive Computer Vision. Kingsley Sage khs20@sussex.ac.uk and Hilary Buxton hilaryb@sussex.ac.uk Prepared under ECVision Specific Action 8-3 http://www.ecvision.org. Course outline. What is Cognitive Computer Vision (CCV) ? Generative models Graphical models
E N D
Cognitive Computer Vision Kingsley Sage khs20@sussex.ac.uk and Hilary Buxton hilaryb@sussex.ac.uk Prepared under ECVision Specific Action 8-3 http://www.ecvision.org
Course outline • What is Cognitive Computer Vision (CCV) ? • Generative models • Graphical models • Techniques for modelling cognitive aspects of CCV • Bayesian inference • Markov Models • Research issues • Coursework and case studies
So what is CCV ? • In this course, we focus on using of ideas from cognitive science and psychology to do CCV • To show how we can build effective CCV systems that are more robust and more capable of solving non-trivial problems than those that do not embrace these ideas • Use statistical inference and machine learning as our tools for modelling cognitively inspired processes • We are not claiming “hard AI” in this course
Key Cognitive Elements • Objects, events, activities and behaviours • “What is it that we are observing?” • Attention and control • “How is it that we observe?”
Key Cognitive Elements • Visual learning and memory • Representation of objects and their behaviour • Recognition • Categorisation • These are “what” problems • Visual control and attention • Perception for tasks using models of expectation • Goals, task context • Resources, embodiment • These are “how” problems • Cognition • From perception to action
Key Cognitive Elements • Visual learning and memory - examples • Learning about objects and how their appearance can change • Recognising activities by the interactions between objects • Extracting invariant models from training data
Learning and “recognising” objects (Murase and Nayar, 1996)
Learn and recognise activities Coupled Hidden Markov Models (CHMM) techniques (Oliver, Rosario & Pentland, 1999) Activities with interactions via coupled states in a HMM
Learning invariant models Means for 3 clusters Variances for 3 clusters
Key Cognitive Elements • Visual control and attention • A framework for attentional control • Inferring likely behaviour using Bayes nets • Deictic markers • Attentional selection of objects
Task Based Control CONTROL POLICY (WITH STATE MEMORY) FEATURE COMBINATION …… dN d1 d2 Image Data Driven A Framework For Task Based Visual Control Scene Interpretation
IGP orient size lo2 ls1 ls2 lo1 BBN Inference of likely vehicle tracks Gong and Buxton, 1993 Fixed camera gives direct set of dependencies Image Grid Position BBN has size/orient hidden nodes Leaf nodes ls1/2, lo1/2 observables
Deictic Markers in inference of behaviour Howarth and Buxton,1996 Left: attention for overtake (overtaken & overtaking vehicle) Right: attention for giveway (stopped & blocker vehicle plus ground-plane conflict zone)
Summary • Cognitive Computer Vision is a multi-disciplinary area of research • Here we use statistical inference and learning for robust models • Task based attentional control is key to prediction and cognitive systems design • Useful reference: “Visual surveillance in a dynamic and uncertain world” Buxton, H and Gong, S, Artificial Intelligence 78, pp 431-459, 1995
Next time … • Generative models • What are they? • Why are they so important to Cognitive Vision?