1 / 99

Motion from image and inertial measurements

Motion from image and inertial measurements. Dennis Strelow Carnegie Mellon University. On the web. Related materials: these and related slides related papers movies VRML models at: http://www.cs.cmu.edu/~dstrelow/epson. Introduction (1). From an image sequence, we can recover:

aideen
Download Presentation

Motion from image and inertial measurements

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Motion from image and inertial measurements Dennis Strelow Carnegie Mellon University

  2. On the web Related materials: • these and related slides • related papers • movies • VRML models at: http://www.cs.cmu.edu/~dstrelow/epson Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 20052

  3. Introduction (1) From an image sequence, we can recover: • 6 degree of freedom (DOF) camera motion • without knowledge of the camera’s surroundings • without GPS Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 20053

  4. Introduction (2) Fitzgibbon Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 20054

  5. Introduction (3) • Potential applications include: • modeling from video Yuji Uchida Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 20055

  6. Introduction (4) • micro air vehicles (MAVs) AeroVironment Black Widow AeroVironment Microbat Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 20056

  7. Introduction (5) • rover navigation Hyperion Nister, et al. Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 20057

  8. Introduction (6) • search and rescue robots Rhex (movies: http://ai.eecs.umich.edu/Rhex/Movies) Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 20058

  9. Introduction (7) • NASA Personal Satellite Assistant (PSA) Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 20059

  10. Introduction (8) For these problems, we want: • 6 DOF motion • in unknown environments • without GPS or other absolute positioning Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200510

  11. Introduction (8) For these problems, we want: • 6 DOF motion • in unknown environments • without GPS or other absolute positioning • using small, light, and cheap sensors Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200511

  12. Introduction (8) For these problems, we want: • 6 DOF motion • in unknown environments • without GPS or other absolute positioning • using small, light, and cheap sensors • over the long term Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200512

  13. Introduction (9) Long-term motion estimation: • absolute distance or time is long • only a small fraction of the scene is visible at any one time Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200513

  14. Introduction (10) • given these requirements, cameras are promising sensors… • …and many algorithms for motion from images already exist Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200514

  15. Introduction (11) But, where are the systems for estimating the motion of: over the long term? Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200515

  16. Introduction (12) …and for automatically modeling • rooms • buildings • cities from a handheld camera? Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200516

  17. Introduction (13) Motion from images suffers from some long-standing difficulties This work attacks these problems by… • exploiting omnidirectional images • exploiting image and inertial measurements • robust image feature tracking • recognizing previously mapped locations Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200517

  18. Outline Motion from images refresher bundle adjustment difficulties Motion from image and inertial measurements Robust image feature tracking Long-term motion estimation Conclusion Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200518

  19. Motion from images: refresher (1) A two-step process is common: • sparse feature tracking • estimation Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200519

  20. Motion from images: refresher (1) A two-step process is common: • sparse feature tracking • estimation Sparse feature tracking: • inputs: raw images • outputs: projections Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200520

  21. Motion from images: refresher (2) Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200521

  22. Motion from images: refresher (3) Template matching: • correlation tracking • Lucas-Kanade (Lucas and Kanade, 1981) Extraction and matching: • Harris features (Harris, 1992) • Scale Invariant Feature Transform (SIFT) keypoints (Lowe, 2004) Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200522

  23. Motion from images: refresher (4) The second step is estimation: • inputs: • projections • outputs: • 6 DOF camera position at the time of each image • 3D position of each tracked point Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200523

  24. Motion from images: refresher (5) Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200524

  25. Motion from images: refresher (6) • bundle adjustment (various, 1950’s) • Kalman filtering (Broida, Chandrashekhar, and Chellappa, 1990) • variable state dimension filter (VSDF) (McLauchlan, 1996) • two- and three-frame methods(Hartley and Zisserman, 2000; Nister, et al. 2004) Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200525

  26. Motion from images: bundle adjustment (1) From tracking, we have the image locations xij for each point j and each image i Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200526

  27. Motion from images: bundle adjustment (2) Suppose we also have estimates of: • the camera rotation ρi and translation ti at time of each image • 3D point positions Xj of each tracked point Then, we can compute reprojections: Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200527

  28. Motion from images: bundle adjustment (3) Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200528

  29. Motion from images: bundle adjustment (4) Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200529

  30. Motion from images: bundle adjustment (5) So, minimize: with respect to all the ρi, ti, Xj Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200530

  31. Motion from images: bundle adjustment (5) So, minimize: with respect to all the ρi, ti, Xj Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200531

  32. Motion from images: difficulties (1) Estimation step can be very sensitive to… • incorrect or insufficient image feature tracking • camera modeling and calibration errors • outlier detection thresholds • sequences with degenerate camera motions Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200532

  33. Motion from images: difficulties (2) Iterative batch methods have poor convergence or may fail to converge if: • observations are missing • the initial estimate is poor Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200533

  34. Motion from images: difficulties (3) Recursive methods suffer from: • poor prior assumptions on the motion • poor approximations in state error modeling Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200534

  35. Motion from images: difficulties (4) Resulting errors are: • gross local errors • long term drift Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200535

  36. Motion from images: difficulties (5) Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200536

  37. Motion from images: difficulties (6) • 151 images, 23 points • manually corrected Lucas-Kanade Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200537

  38. Motion from images: difficulties (7) • squares: ground truth points • dash-dotted line: accurate estimate • solid line: image-only, bundle adjustment estimate Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200538

  39. Outline Motion from images Motion from image and inertial measurements inertial sensors algorithms and results related work Robust image feature tracking Long-term motion estimation Conclusion Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200539

  40. Motion from image and inertial measurements: inertial sensors (1) • inertial sensors can be integrated to estimate six degree of freedom motion Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200540

  41. Motion from image and inertial measurements: inertial sensors (2) But many applications require small, light, and cheap sensors Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200541

  42. Motion from image and inertial measurements: inertial sensors (3) Integrating the outputs of these low grade sensors will produce drifting motion because of: • noise • unmodeled nonlinearities Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200542

  43. Motion from image and inertial measurements: inertial sensors (4) • And, we can’t even integrate until we can separate the effects of… • rotation ρ • gravity g • acceleration a • slowly changing bias ba • noise n • …in the accelerometer measurements Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200543

  44. Motion from image and inertial measurements: inertial sensors (5) Image and inertial measurements are highly complementary With inertial measurements we can: • decrease sensitivity in image-only estimates • establish two rotation angles without drift • establish the global scale Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200544

  45. Motion from image and inertial measurements: inertial sensors (5) Image and inertial measurements are highly complementary With inertial measurements we can: • decrease sensitivity in image-only estimates • establish two rotation angles without drift • establish the global scale …even with our low-grade sensors Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200545

  46. Motion from image and inertial measurements: inertial sensors (6) With image measurements, we can: • reduce the drift in integrating inertial data • distinguish between… • rotation ρ • gravity g • acceleration a • bias ba • noise n …in accelerometer measurements Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200546

  47. Motion from image and inertial measurements: algorithms and results (1) This work has developed both: • batch • recursive algorithms for motion from image and inertial measurements Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200547

  48. Motion from image and inertial measurements: algorithms and results (2) Gyro measurements: • ω’, ω: measured and actual angular velocity • bω: gyro bias • n: gaussian noise Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200548

  49. Motion from image and inertial measurements: algorithms and results (3) Accelerometer measurements: • ρ: rotation • a’, a: measured and actual acceleration • g: gravity vector • ba: accelerometer bias • n: gaussian noise Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200549

  50. Motion from image and inertial measurements: algorithms and results (4) • batch algorithm minimizes a combined error: Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200550

More Related