1 / 61

An Object Tacking Paradigm with Active Appearance Models for Augmented Reality

An Object Tacking Paradigm with Active Appearance Models for Augmented Reality. Presented by Pat Chan Pik Wah 28/04/2005 Qualifying Examination. Outline. Research Objective Introduction Augmented Reality Object Tracking Active Appearance Models (AAMs) Proposed Object Tracking Paradigm

rufus
Download Presentation

An Object Tacking Paradigm with Active Appearance Models for Augmented Reality

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. An Object Tacking Paradigm with Active Appearance Models for Augmented Reality Presented by Pat Chan Pik Wah 28/04/2005 Qualifying Examination

  2. Outline • Research Objective • Introduction • Augmented Reality • Object Tracking • Active Appearance Models (AAMs) • Proposed Object Tracking Paradigm • Paradigm Architecture • Experiments • Research Issues • Conclusion

  3. Research Objective • Object tracking is an essential component for Augmented Reality. • There is a lack of good object tracking paradigm. • Active Appearance Models is promising. • Propose a new object tracking paradigm with AAMs in order to provide a real-time and accurate registration for Augmented Reality. • Nature of the paradigm: • Effective • Accurate • Robust

  4. Augmented Reality • An Augmented Reality system supplements the real world with virtual objects that appear to coexist in the same space as the real world • Properties : • Combine real and virtual objects in a real environment • Runs interactively, and in real time • Registers(aligns) real and virtual objects with each other

  5. Augmented Reality • Projects related to AR

  6. Augmented Reality • Display • Presenting virtual objects on real environment • Tracking • Following user’s and virtual object’s movements by means of a special device or techniques • 3D Modeling • Forming virtual object • Registration • Blending real and virtual objects

  7. edges, corners, lines, curves, and color regions Pixels Object Tracking • Visual content can be modeled as a hierarchy of abstractions. • At the first level are the raw pixels with color or brightness information. • Further processing yields features such as edges, corners, lines, curves, and color regions. • A higher abstraction layer may combine and interpret these features as objects and their attributes. Object

  8. Object Tracking • Accurately tracking the user’s position is crucial for AR registration • The objective is to obtain an accurate estimate of the position (x,y) of the object tracked • Tracking = correspondence + constraints + estimation • Based on reference image of the object, or properties of the objects. • Two main stages for tracking object in video: • Isolation of objects from background in each frames • Association of objects in successive frames in order to trace them

  9. Object Tracking • Object Tracking can be briefly divides into following stages: • Input (object and camera) • Detecting the Objects • Motion Estimation • Corrective Feedback • Occlusion Detection

  10. Object Tracking • Expectation Maximization • Find the local maximum likelihood solution • Some variables are hidden or incomplete • Kalman Filter • Optimal linear predict the state of a model • Condensation • Combines factored sampling with learned dynamical models • propagate an entire probability of object position and shape

  11. Object Tracking • Pervious Work : • Marker-based Tracking • Feature-based Tracking • Template-based object tracking • Correlation-based tracking • Change-based tracking • 2D layer tracking • tracking of articulated objects

  12. Pervious Work • Marker-based Tracking • Marker-less based Tracking • Feature-based Tracking • Shape-based approaches • Color-based approaches

  13. Pervious Work • Template-based object tracking • Fixed template matching • Image subtraction • Correlation • Deformable template matching

  14. Pervious Work • Object tracking using motion information • Motion-based approaches • Model-based approaches • Boundary-based approaches • Snakes • Geodesic active contour models • Region-based approaches

  15. Active Appearance Models • The Active Appearance Model (AAM) algorithm is a powerful tool for modeling images of deformable objects. • AAM combines a subspace-based deformable model of an object’s appearance • Fit the model to a previously unseen image.

  16. Timeline for development of AAMs and ASMs

  17. Active Appearance Models (AAMs) • 2D linear shape is defined by 2D triangulated mesh and in particular the vertex locations of the mesh. • Shape scan be expressed as a base shape s0. • pi are the shape parameter. • s0 is the mean shape and the matrices si are the eigenvectors corresponding to the m largest eigenvalues

  18. A0(u) A1(u) A2(u) A3(u) Active Appearance Models (AAMs) • The appearanceof an independent AAM is defined within the base mesh s0. A(u) defined over the pixels u∈ s0 • A(u) can be expressed as a base appearance A0(u) plus a linear combination of l appearance • Coefficients λi are the appearance parameters.

  19. Active Appearance Models (AAMs) • The AAM model instance with shape parameters pand appearance parameters λ is then created by warping the appearance Afrom the base mesh s0 to the model shape s. Piecewise affine warp W(u; p): (1) for any pixel u in s0 find out which triangle it lies in, (2) warp u with the affine warp for that triangle. M(W(u;p))

  20. u u u u Fitting AAMs • Minimize the error between I (u) and M(W(u; p)) = A(u). • If u is a pixel in s0, then the corresponding pixel in the input image I is W(u; p). • At pixel u the AAM has the appearance • At pixel W(u; p), the input image has the intensity I (W(u; p)). • Minimize the sum of squares of the difference between these two quantities:

  21. DEMO Video – 2D AAMs

  22. DEMO Video – 2D AAMs

  23. Recent Work for Improving AAMs • Combine 2D+3D AAMs

  24. Combined 2D + 3D AAMs • At time t, we have • 2D AAM shape vector in all N images into a matrix: • Represent as a 3D linear shape modesW = MB =

  25. Compute the 3D Model AAM shapes AAM appearance First three 3D shapes modes

  26. Constraining an AAM with 3D Shape • Constraints on the 2D AAM shape parameters p = (p1, … , pm) that force the AAM to only move in a way that is consistent with the 3D shape modes: • and the 2D shape variation of the 3D shape modes over all imaging condition is: • Legitimate values of P and p such that the 2D projected 3D shape equals the 2D shape of AAM. The constraint is written as:

  27. An Object Tacking Paradigm with Active Appearance Models

  28. Training Images Proposed Object Tracking Paradigm Paradigm Architecture Occlusion Detection Training Active Appearance Model Video • Shape Model • Appearance Model Motion Modeling Initialization Kalman Filter

  29. Steps in Object Tracking Paradigm • Preporcessing • Training the Active Appearance Model. • Get the shape model and the appearance model for the object to be tracked. • Initialization • Locating the object position in the video. • In our scheme, we make use of AAMs. • Motion Modeling • Estimate the motion of the object • Modeling the AAMs as a problem in the Kalman filter to perform the prediction. • Occlusion Detection • Preventing the lost of position of the object by occluding of other objects.

  30. Enhancing Active Appearance Models • Shape • Appearance • Combine the shape and the appearance parameters for optimization • In video, shape and appearance may not enough, there are many characteristics and features, such as lightering, brightness, etc… L=[L1, L2, ……, Lm]T

  31. Iterative Search for Fitting Active Appearance Model

  32. Iterative Search for Fitting Active Appearance Model • Can be improved by: • Prediction matrix • Searching space

  33. Initialization for AAMs

  34. Motion Modeling • Initial estimate in a frame should be better predicted than just the adaptation from the previous frame. • Can be achieved by motion estimation • AAMs can do the modeling part • Kalman filter can do the prediction part

  35. Kalman Filter • Adaptive filter • Model the state of a discrete dynamic system. • Originally developed in 1960 • Filter out noise in electronic signals.

  36. Kalman Filter • Formally, we have the model • For our tracking system,

  37. Kalman Filter

  38. Occlusion Detection • WHY? • Positioning of objects • To perform cropping • When a real object overlays a virtual one, the virtual object should be cropped before the overlay • HOW? • High resolution and sharp object boundaries • Right occluding boundaries of objects • Camera matrix for video capturing

  39. Training Images Proposed Object Tracking Paradigm Paradigm Architecture Occlusion Detection Training Active Appearance Model Video • Shape Model • Appearance Model Active Appearance Model Fitting Initialization Kalman Filter

  40. Experimental Setup • AAM-api from DTU • OpenCV • Pentium 4 CPU 2.00GHz and 512MB RAM

  41. Experiment on AAMs (1) • Training Image

  42. Experiment on AAMs (1) Texture Shape

  43. Experiment on AAMs (1) After optimized Initialization

  44. Demo Video

  45. Demo Video

  46. Demo Video

  47. Demo Video

  48. Experiment on AAMs (2) • Training Images

  49. Experiment on AAMs Texture Shape

  50. Experiment on AAMs • Trapped in local minimum After optimized Initialization

More Related