Dynamic Data Analysis Projects in the Image Analysis and Motion Capture L abs

1. NSF I/UCRC Workshop Problem definition Dynamic Data Analysis Projects in the Image Analysis and Motion Capture Labs In dynamic 3d data non-rigid registration is essential for 3d surface tracking, expression analysis and transfer, dense motion capture data processing etc. The deformation error can be measured in the 2D domain using conformal mapping and three correspondences, leading to a high-order graph matching problem Non-rigid Surface Registration Using High-Order Graph Matching Sparse and Locally Constant Gaussian Graphical Models Examples Local constancy: Goal: We want to explore the structure of probabilistic relationships in massive spatiotemporal datasets. We want to learn sparse Gaussian graphical models, while enforcing spatial coherence of the dependence and independence relationships. Such learned structure permits efficient inference but also gives insights into the nature of the data Current registration results: Tracking subtle details: Strictly Concave Penalized Maximum Likelihood: We perform maximum likelihood estimation with sparseness and local constancy priors = + Challenges: - original data are not registered in object space and the points may have different motion vectors and velocities) - The large size of the datasets (tens of thousands of 3d points per frame) require accurate and efficient processing Expression transfer: 3 Example Applications: Figure: cardiac MRI displacement and the corresponding spatial manifold sparseness penalty local constancy penalty log-likelihood of the dataset Results regularization parameters precision matrix for N variables sample covariance matrix discrete derivative operator for M spatial neighborhood relationships Statistical Shadow and Illumination Estimation for Real-World Images Results Goal: Estimate the Illumination environment from a single image, with rough knowledge of the 3D geometry and in the presence of texture Figure: manually labeled walking sequence. Right: leg/leg, hand/leg interaction in red, independent leg motion in blue Figure: functional brain MRI of a monetary reward task; left: 16 cocaine subjects, more connections in the cerebellum (green); right: 12 control subjects, more connections in the prefrontal cortex(red) A novel cue for shading/shadow extraction Illumination from Caltech 101 motorbike images, using a common 3D model for the whole class: Figure: local constancy in a 2D dataset (spatial neighborhood in black dashed lines) - local constancy does not discourage long range interactions Simultaneous Analysis of Facial Expression and EEG Data • Goal: Examine facial expressions related to drug craving and drug addiction • Dataset: Videos of the facial expressions of subjects and simultaneously captured EEG (electroencephalogram) data • The subjects watch a series of images belonging in several categories (happy, unpleasant, drugs, neutral) • Method: • Facial expression features are tracked using an Active Appearance Model (AAM) • FACS (Facial Action Coding System) codes are retrieved from the feature movement An MRF model for robust illumination estimation • Models the creation of cast shadows in a statistical framework • Allows estimation of the illumination from real images, modeling objects with bounding boxes or general class geometry Applications: integration in scene understanding, search in large image databases, augmented reality etc Figure: cross-validated log-likelihood on the testing set *not statistically significantly different from our method