Video Mining

1. Video Mining Learning Patterns of Behaviour via an Intelligent Image Analysis System

2. Introduction • All large archives of video are now available in repositories, there is hidden much potentially useful knowledge Large archives of videos Data mining Techniques Latent, Useful, Interesting Information Jang Hee Kyun

3. Introduction • Video mining is more challenging, due to lack of explicit structure in the raw data in video archives • Ex) Surveillance, Analysis of expert’s activities Studying animal behaviour, Emulate robot behaviour • Each applications there are often patterns of activity • They can be classified in order to gain more general insights into agent and object movements and behaviour Jang Hee Kyun

4. Video Mining • Interesting in that the raw data in videos is expressed in a way that is not directly amenable to the use of conventional mining techniques • There is a lot of variation in the details of the patterns and it is important to abstract out the key features of a behaviour • Unwanted or irrelevant details and noise can be filtered out Jang Hee Kyun

5. Video Mining • Video mining of patterns of behaviour and their inter-play, has to deal with very dynamic situations • Techniques that have been developed for the extraction of temporal rules from collections of time series data • We now want to identify patterns of data that are unusual, and discover inter-relationships between the patterns of different agents and objects Jang Hee Kyun

6. Discover Rules • First Stream • Detect ‘abnormal ‘ or ‘interesting’ behaviour • The ability to learn what is ‘abnormal’ or ‘ interesting ’ • Ultimate goal is that they will use only innate knowledge • Second Stream • Summarisation for behavioural pattern detection/ matching in the second stream using AI (Artificial Intelligence) and DM (Data Mining) algorithms for time series analysis Jang Hee Kyun

7. Discover Rules • It will be able to operate without the direct intervention of a user, and be able to control its own focus of attention to some extent • This will in turn influence how it operates in related situations in the future Jang Hee Kyun

8. Background and approach • We use our own system, ModTrack, for vehicle detection and tracking • “Independent Moving Object Detection Using a Colour Background Model” by F. Campbell – West, P. Miller, H. Wang • DM (Data Mining) aspects • Tracking system • Identify abnormal behaviour • Infer unusual pattern of activity • AI (Artificial Intelligence) aspects • Learning how another agent learns and making use of the results Jang Hee Kyun

9. Method of representation and analysis • Our objective is to reverse engineer what we observe in the real world by using a vision or imaging system • We need to emulate the behaviour of the real world actors • How does a robot adjust its knowledge about the behaviour of a light using the adaptive learning paradigm? Jang Hee Kyun

10. Method of representation and analysis • We use the robot’s intention not only as a consideration for our decision making, but also as a guide for our accumulation of observations • We make a qualitative assessment by distinguishing suggestion and confirmation Jang Hee Kyun

11. Example Jang Hee Kyun

12. Method of representation and analysis • As it detects sequences of such atomic movements the system records them • Behaviour pattern detection • Classify these behavioural seuqences • Classifier is important requirement • Represent activities rule set • Behaviour matching and prediction Jang Hee Kyun

13. Conclusions and Summary • ModTrack was used to obtain the behavioural traces of the robot/agent • Using representation we can build a new representation of what is happening with the raw data • We have shown how detailed behaviours from video can be coarsened and mined to obtain useful knowledge Jang Hee Kyun