1 / 26

The Brightness Constraint

=. -. I. I. (. x. ,. y. ). J. (. x. ,. y. ). Where:. t. Insufficient info. The Brightness Constraint. Brightness Constancy Equation:. Linearizing (assuming small (u,v) ):. Each pixel provides 1 equation in 2 unknowns (u,v).

gburns
Download Presentation

The Brightness Constraint

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. = - I I ( x , y ) J ( x , y ) Where: t Insufficient info. The Brightness Constraint Brightness Constancy Equation: Linearizing (assuming small (u,v)): Each pixel provides 1 equation in 2 unknowns (u,v). Another constraint:Global Motion Model Constraint

  2. Global Motion Models 2D Models: • Affine • Quadratic • Planar projective transform (Homography) 3D Models: • Rotation, Translation, 1/Depth • Instantaneous camera motion models • Plane+Parallax

  3. Least Square Minimization (over all pixels): Example: Affine Motion Substituting into the B.C. Equation: Each pixel provides 1 linear constraint in 6 global unknowns • (minimum 6 pixels necessary)

  4. Coarse-to-Fine Estimation Jw refine warp + u=1.25 pixels u=2.5 pixels ==> small u and v ... u=5 pixels u=10 pixels image J image J image I image I Pyramid of image J Pyramid of image I

  5. Quadratic– instantaneous approximation to planar motion Other 2D Motion Models Projective – exact planar motion (Homography)

  6. Generated Mosaic image Panoramic Mosaic Image Original video clip Alignment accuracy (between a pair of frames): error < 0.1 pixel

  7. Video Removal Original Original Outliers Synthesized

  8. Video Enhancement ORIGINAL ENHANCED

  9. Direct Methods: Methods for motion and/or shape estimation, which recover the unknown parameters directly from measurable image quantities at each pixel in the image. Minimization step: Direct methods: Error measure based on dense measurable image quantities(Confidence-weighted regression; Exploits all available information) Feature-based methods: Error measure based on distances of a sparse set of distinct features.

  10. Benefits of Direct Methods • Subpixel accuracy. • Does not need distinct features. • Locking property.

  11. Limitations • Limited search range (up to 10% of the image size). • Brightness constancy.

  12. Handling Varying Brightness Preprocessing: • Mean and contrast normalization. • Laplacian pyramids. Measurable image quantities: • brightness values • correlation surfaces [Irani-Anandan:iccv98, Mandelbaum-et-al:iccv99] • mutual information [Viola-et-al]

  13. Video Indexing and Editing

  14. The 2D/3D Dichotomy Camera motion + Scene structure + Independent motions Camera induced motion = + Independent motions = Image motion = 2D techniques 3D techniques Do not model “3D scenes” Singularities in “2D scenes”

  15. The residual parallax lies on aradial (epipolar) field: epipole The Plane+Parallax Decomposition Original Sequence Plane-Stabilized Sequence

  16. Benefits of the P+P Decomposition 1. Reduces the search space: • Eliminates effects of rotation • Eliminates changes in camera parameters / zoom • Camera parameters: Need to estimate only epipole. • (gauge ambiguity: unknown scale of epipole) • Image displacements: Constrained to lie on radial lines • (1-D search problem) A result of aligning an existing structure in the image.

  17. Benefits of the P+P Decomposition 2. Scene-Centered Representation: Translation or pure rotation ??? Focus on relevant portion of info Remove global component which dilutes information !

  18. Benefits of the P+P Decomposition 2. Scene-Centered Representation: Shape =Fluctuations relative to a planar surface in the scene STAB_RUG SEQ

  19. total distance [97..103] camera center scene global (100) component local [-3..+3] component Benefits of the P+P Decomposition 2. Scene-Centered Representation: Shape =Fluctuations relative to a planar surface in the scene • Height vs. Depth (e.g., obstacle avoidance) • Appropriate units for shape • A compact representation • - fewer bits, progressive encoding

  20. Benefits of the P+P Decomposition 3. Stratified 2D-3D Representation: • Start with 2D estimation (homography). • 3D info builds on top of 2D info. Avoids a-priori model selection.

  21. Dense 3D Reconstruction(Plane+Parallax) Original sequence Plane-aligned sequence Recovered shape

  22. Dense 3D Reconstruction(Plane+Parallax) Original sequence Plane-aligned sequence Recovered shape

  23. Results Original sequence Plane-aligned sequence Recovered shape

  24. p Epipolar line epipole Brightness Constancy constraint Multi-Frame vs. 2-Frame Estimation 1. Eliminating Aperture Problem The intersection of the two line constraints uniquely defines the displacement.

  25. other epipolar line p Epipolar line another epipole epipole Brightness Constancy constraint Multi-Frame vs. 2-Frame Estimation 1. Eliminating Aperture Problem The other epipole resolves the ambiguity ! The two line constraints are parallel ==> do NOT intersect

  26. Instantaneous camera motion: Global parameters: Local Parameter: Residual Planar Parallax Motion Global parameters: Local Parameter: 3D Motion Models

More Related