320 likes | 436 Views
Computer Vision and Media Group: Selected Previous Work. David Gibson, Neill Campbell Colin Dalton Department of Computer Science University of Bristol. Duck: The Automatic Generation of 3D Models. Generating 3D computer models is difficult Put object on turntable
E N D
Computer Vision and Media Group:Selected Previous Work David Gibson, Neill Campbell Colin Dalton Department of Computer Science University of Bristol AutoArch Overview
Duck: The AutomaticGeneration of 3D Models • Generating 3D computer models is difficult • Put object on turntable • Take 8 pictures of it from different angles • Crank the handle… • No skilled user or expensive equipment • Make avatars by spinning person on chair AutoArch Overview
Cog and Stepper • Automatically inject ‘life’ into computer animations • 3D swathe through 4D space time • Where space is 3D computer model • Or just to make things look strange! AutoArch Overview
Casablanca: Motion Ripper • Computer animation driven by film • Animator labels a small number of points • System then tracks these points over all frames • Motions are extracted and used to drive animation AutoArch Overview
Laughing ManMotion Ripper Part 2 • Automatic video creation • Points are marked and tracked • System learns the motions • System generates new motions which are different but ‘correct’ • Forever! AutoArch Overview
AutoArch: The Automatic Archiving of Wildlife Film Footage David Gibson, Neill Campbell David Tweed, Sarah Porter Department of Computer Science University of Bristol AutoArch Overview
Motivation • BBC Natural History Unit • Manual archiving/meta data generation • Reuse problematic • Inefficient/time consuming • Expensive • Limited access • Obvious need to automate AutoArch Overview
Objectives • Generate efficient visual representations • Video segmentation • Visual browsing/summarisation • Visual searching • Generate as much meta data automatically • Camera motions/effects • Scene structure • Scene content AutoArch Overview
Visualisation and Searching Visualisation based algorithms Shot Segmentation Visual Summarisation Motion Analysis Colour/Texture Analysis Meta data extraction algorithms Catalogue Entry System Overview AutoArch Overview
Video Segmentation AutoArch Overview
Visual Summarisation • Key frame extraction AutoArch Overview
Entire shot Visual Summarisation Tree Level of detail AutoArch Overview
Visual Searching • Layered 2D representation of high D clip space AutoArch Overview
Motion Analysis using point tracking • Camera Motion Estimation • Event/Area of Interest Detection • Gait Analysis • Foreground/Background Separation • Combine with Colour and Texture for Classification • See cheetah track avi AutoArch Overview
Camera Pan BCD0111.09_0085.epslines = 47, curls = 98, shorts = 5long lines = 47, mode = 95.00, mean = 95.21, std = 4.15zoom centre = (603.01, 63.65), val = -0.2356zoom residual per line = 22.92zoom residual #2 per line = 28.92Average line vector: 109.94 -8.27pan/tilt angle: 94.30, vector: (109.94 -8.27)pan/tilt residual per line = 21.67pan/tilt residual #2 per line = 33.38percentage of lines within 5% of mode: 89.36 AutoArch Overview
Camera Zoom BCD0113.15_0067.epslines = 142, curls = 1, shorts = 7long lines = 134, mode = 340.00, mean = 227.24, std = 128.76zoom centre = (182.97, 55.52), val = 0.2063zoom residual per line = 4.86zoom residual #2 per line = 6.90Average line vector: -3.81 17.28pan/tilt angle: 347.57, vector: (-3.81 17.28)pan/tilt residual per line = 13.85pan/tilt residual #2 per line = 16.13percentage of lines within 5% of mode: 17.16 AutoArch Overview
Tracking Failure This could be an interesting event in its self: flocking, herding, close up of lots of activity, shot grouping, etc. AutoArch Overview
Event/Area of InterestDetection AutoArch Overview
Frequency Analysis:Gait Detection After trajectory segmentation FFT AutoArch Overview
Foreground model Feature space #2 Background model Feature space #1 Foreground/BackgroundExtraction Which pixels are foreground? AutoArch Overview
Animal Identification Give models a name: = zebra = cheetah = lion = elephant AutoArch Overview
Some Problems • Noise in images • Noise in measurements • Camouflage • Occlusion • Answer: Need higher level models • See next few slides AutoArch Overview
Model Based Tracking AutoArch Overview
Lion Tracking • Synchronise horse model with lion points • Move and deform horse model to lion points • See avi • To do: Improve spatial deformation, especially for legs, using colour and texture AutoArch Overview
Multiple Object Tracking AutoArch Overview
Conclusions • Visualisation is very powerful • Combined with text is even better! • Assists searching and communication • Lots of meta data can be auto generated • Assists archiving • Help to prioritise manual archiving • Can be applied to any visual media AutoArch Overview