1 / 110

Motion from image and inertial measurements

Motion from image and inertial measurements. Dennis Strelow Honeywell Advanced Technology Lab. On the web. Related materials: these slides related papers movies VRML models at: http://www.dennis-strelow.com/umn. Introduction (1). From an image sequence, we can recover:

brooke-vance
Download Presentation

Motion from image and inertial measurements

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Motion from image and inertial measurements Dennis Strelow Honeywell Advanced Technology Lab

  2. On the web Related materials: • these slides • related papers • movies • VRML models at: http://www.dennis-strelow.com/umn Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 20062

  3. Introduction (1) From an image sequence, we can recover: • 6 degree of freedom (DOF) camera motion • without knowledge of the camera’s surroundings • without GPS Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 20063

  4. Introduction (2) Fitzgibbon Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 20064

  5. Introduction (3) • Potential applications include: • modeling from video Yuji Uchida Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 20065

  6. Introduction (4) • micro air vehicles (MAVs) AeroVironment Black Widow AeroVironment Microbat Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 20066

  7. Introduction (5) • rover navigation Hyperion Nister, et al. Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 20067

  8. Introduction (6) • search and rescue robots Rhex (movies: http://ai.eecs.umich.edu/Rhex/Movies) Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 20068

  9. Introduction (7) • NASA Personal Satellite Assistant (PSA) Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 20069

  10. Introduction (8) For these problems, we want: • 6 DOF motion • in unknown environments • without GPS or other absolute positioning Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200610

  11. Introduction (8) For these problems, we want: • 6 DOF motion • in unknown environments • without GPS or other absolute positioning • using small, light, and cheap sensors Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200611

  12. Introduction (8) For these problems, we want: • 6 DOF motion • in unknown environments • without GPS or other absolute positioning • using small, light, and cheap sensors • over the long term Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200612

  13. Introduction (9) Long-term motion estimation: • absolute distance or time is long • only a small fraction of the scene is visible at any one time Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200613

  14. Introduction (10) • given these requirements, cameras are promising sensors… • …and many algorithms for motion from images already exist Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200614

  15. Introduction (11) But, where are the systems for estimating the motion of: over the long term? Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200615

  16. Introduction (12) …and for automatically modeling • rooms • buildings • cities from a handheld camera? Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200616

  17. Introduction (13) Motion from images suffers from some long-standing difficulties This work attacks these problems by… • exploiting image and inertial measurements • robust image feature tracking • recognizing previously mapped locations • exploiting omnidirectional images Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200617

  18. Outline Motion from images refresher bundle adjustment difficulties Motion from image and inertial measurements Robust image feature tracking Long-term motion estimation Conclusion Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200618

  19. Motion from images: refresher (1) A two-step process is common: • sparse feature tracking • estimation Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200619

  20. Motion from images: refresher (1) A two-step process is common: • sparse feature tracking • estimation Sparse feature tracking: • inputs: raw images • outputs: projections Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200620

  21. Motion from images: refresher (2) Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200621

  22. Motion from images: refresher (3) Template matching: • correlation tracking • Lucas-Kanade (Lucas and Kanade, 1981) Extraction and matching: • Harris features (Harris, 1992) • Scale Invariant Feature Transform (SIFT) keypoints (Lowe, 2004) Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200622

  23. Motion from images: refresher (4) The second step is estimation: • inputs: • projections • outputs: • 6 DOF camera position at the time of each image • 3D position of each tracked point Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200623

  24. Motion from images: refresher (5) Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200624

  25. Motion from images: refresher (6) • bundle adjustment (various, 1950’s) • Kalman filtering (Broida, Chandrashekhar, and Chellappa, 1990) • variable state dimension filter (VSDF) (McLauchlan, 1996) • two- and three-frame methods(Hartley and Zisserman, 2000; Nister, et al. 2004) Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200625

  26. Motion from images: bundle adjustment (1) From tracking, we have the image locations xij for each point j and each image i Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200626

  27. Motion from images: bundle adjustment (2) Suppose we also have estimates of: • the camera rotation ρi and translation ti at time of each image • 3D point positions Xj of each tracked point Then, we can compute reprojections: Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200627

  28. Motion from images: bundle adjustment (3) Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200628

  29. Motion from images: bundle adjustment (4) Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200629

  30. Motion from images: bundle adjustment (5) So, minimize: with respect to all the ρi, ti, Xj Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200630

  31. Motion from images: bundle adjustment (5) So, minimize: with respect to all the ρi, ti, Xj Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200631

  32. Motion from images: difficulties (1) Estimation step can be very sensitive to… • incorrect or insufficient image feature tracking • camera modeling and calibration errors • outlier detection thresholds • sequences with degenerate camera motions Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200632

  33. Motion from images: difficulties (2) Iterative batch methods have poor convergence or may fail to converge if: • observations are missing • the initial estimate is poor Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200633

  34. Motion from images: difficulties (3) Recursive methods suffer from: • poor prior assumptions on the motion • poor approximations in state error modeling Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200634

  35. Motion from images: difficulties (4) Resulting errors are: • gross local errors • long term drift Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200635

  36. Motion from images: difficulties (5) Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200636

  37. Motion from images: difficulties (6) • 151 images, 23 points • manually corrected Lucas-Kanade Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200637

  38. Motion from images: difficulties (7) • squares: ground truth points • dash-dotted line: accurate estimate • solid line: image-only, bundle adjustment estimate Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200638

  39. Outline Motion from images Motion from image and inertial measurements inertial sensors algorithms and results related work Robust image feature tracking Long-term motion estimation Conclusion Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200639

  40. Motion from image and inertial measurements: inertial sensors (1) • inertial sensors can be integrated to estimate six degree of freedom motion Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200640

  41. Motion from image and inertial measurements: inertial sensors (2) But many applications require small, light, and cheap sensors Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200641

  42. Motion from image and inertial measurements: inertial sensors (3) Integrating the outputs of these low grade sensors will produce drifting motion because of: • noise • unmodeled nonlinearities Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200642

  43. Motion from image and inertial measurements: inertial sensors (4) • And, we can’t even integrate until we can separate the effects of… • rotation ρ • gravity g • acceleration a • slowly changing bias ba • noise n • …in the accelerometer measurements Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200643

  44. Motion from image and inertial measurements: inertial sensors (5) Image and inertial measurements are highly complementary With inertial measurements we can: • decrease sensitivity in image-only estimates • establish two rotation angles without drift • establish the global scale Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200644

  45. Motion from image and inertial measurements: inertial sensors (5) Image and inertial measurements are highly complementary With inertial measurements we can: • decrease sensitivity in image-only estimates • establish two rotation angles without drift • establish the global scale …even with our low-grade sensors Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200645

  46. Motion from image and inertial measurements: inertial sensors (6) With image measurements, we can: • reduce the drift in integrating inertial data • distinguish between… • rotation ρ • gravity g • acceleration a • bias ba • noise n …in accelerometer measurements Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200646

  47. Motion from image and inertial measurements: algorithms and results (1) This work has developed both: • batch • recursive algorithms for motion from image and inertial measurements Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200647

  48. Motion from image and inertial measurements: algorithms and results (2) Gyro measurements: • ω’, ω: measured and actual angular velocity • bω: gyro bias • n: gaussian noise Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200648

  49. Motion from image and inertial measurements: algorithms and results (3) Accelerometer measurements: • ρ: rotation • a’, a: measured and actual acceleration • g: gravity vector • ba: accelerometer bias • n: gaussian noise Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200649

  50. Motion from image and inertial measurements: algorithms and results (4) • batch algorithm minimizes a combined error: Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 200650

More Related