1 / 114

Motion from image and inertial measurements

Motion from image and inertial measurements. Dennis Strelow Carnegie Mellon University. On the web. Related materials: these slides related papers movies VRML models at: http://www.cs.cmu.edu/~dstrelow/northrop2005. Introduction (1). From an image sequence, we can recover:

kyle
Download Presentation

Motion from image and inertial measurements

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Motion from image and inertial measurements Dennis Strelow Carnegie Mellon University

  2. On the web Related materials: • these slides • related papers • movies • VRML models at: http://www.cs.cmu.edu/~dstrelow/northrop2005 Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 20042

  3. Introduction (1) From an image sequence, we can recover: • 6 degree of freedom (DOF) camera motion • without knowledge of the camera’s surroundings • without GPS Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 20043

  4. Introduction (2) Fitzgibbon Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 20044

  5. Introduction (3) • Potential applications include: • modeling from video Yuji Uchida Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 20045

  6. Introduction (4) • micro air vehicles (MAVs) AeroVironment Black Widow AeroVironment Microbat Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 20046

  7. Introduction (5) • rover navigation Hyperion Nister, et al. Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 20047

  8. Introduction (6) • search and rescue robots Rhex (movies: http://ai.eecs.umich.edu/Rhex/Movies) Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 20048

  9. Introduction (7) • NASA Personal Satellite Assistant (PSA) Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 20049

  10. Introduction (8) For these problems, we want: • 6 DOF motion • in unknown environments • without GPS or other absolute positioning Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200410

  11. Introduction (8) For these problems, we want: • 6 DOF motion • in unknown environments • without GPS or other absolute positioning • using small, light, and cheap sensors Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200411

  12. Introduction (8) For these problems, we want: • 6 DOF motion • in unknown environments • without GPS or other absolute positioning • using small, light, and cheap sensors • over the long term Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200412

  13. Introduction (9) Long-term motion estimation: • absolute distance or time is long • only a small fraction of the scene is visible at any one time Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200413

  14. Introduction (10) • given these requirements, cameras are promising sensors… • …and many algorithms for motion from images already exist Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200414

  15. Introduction (11) But, where are the systems for estimating the motion of: over the long term? Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200415

  16. Introduction (12) …and for automatically modeling • rooms • buildings • cities from a handheld camera? Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200416

  17. Introduction (13) Motion from images suffers from some long-standing difficulties This work attacks these problems by… • exploiting image and inertial measurements • robust image feature tracking • recognizing previously mapped locations • exploiting omnidirectional images Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200417

  18. Outline Motion from images refresher bundle adjustment difficulties Motion from image and inertial measurements Robust image feature tracking Long-term motion estimation Conclusion Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200418

  19. Motion from images: refresher (1) A two-step process is common: • sparse feature tracking • estimation Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200419

  20. Motion from images: refresher (1) A two-step process is common: • sparse feature tracking • estimation Sparse feature tracking: • inputs: raw images • outputs: projections Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200420

  21. Motion from images: refresher (2) Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200421

  22. Motion from images: refresher (3) Template matching: • correlation tracking • Lucas-Kanade (Lucas and Kanade, 1981) Extraction and matching: • Harris features (Harris, 1992) • Scale Invariant Feature Transform (SIFT) keypoints (Lowe, 2004) Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200422

  23. Motion from images: refresher (4) The second step is estimation: • inputs: • projections • outputs: • 6 DOF camera position at the time of each image • 3D position of each tracked point Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200423

  24. Motion from images: refresher (5) Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200424

  25. Motion from images: refresher (6) • bundle adjustment (various, 1950’s) • Kalman filtering (Broida, Chandrashekhar, and Chellappa, 1990) • variable state dimension filter (VSDF) (McLauchlan, 1996) • two- and three-frame methods(Hartley and Zisserman, 2000; Nister, et al. 2004) Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200425

  26. Motion from images: bundle adjustment (1) From tracking, we have the image locations xij for each point j and each image i Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200426

  27. Motion from images: bundle adjustment (2) Suppose we also have estimates of: • the camera rotation ρi and translation ti at time of each image • 3D point positions Xj of each tracked point Then, we can compute reprojections: Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200427

  28. Motion from images: bundle adjustment (3) Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200428

  29. Motion from images: bundle adjustment (4) Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200429

  30. Motion from images: bundle adjustment (5) So, minimize: with respect to all the ρi, ti, Xj Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200430

  31. Motion from images: bundle adjustment (5) So, minimize: with respect to all the ρi, ti, Xj Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200431

  32. Motion from images: difficulties (1) Estimation step can be very sensitive to… • incorrect or insufficient image feature tracking • camera modeling and calibration errors • outlier detection thresholds • sequences with degenerate camera motions Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200432

  33. Motion from images: difficulties (2) Iterative batch methods have poor convergence or may fail to converge if: • observations are missing • the initial estimate is poor Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200433

  34. Motion from images: difficulties (3) Recursive methods suffer from: • poor prior assumptions on the motion • poor approximations in state error modeling Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200434

  35. Motion from images: difficulties (4) Resulting errors are: • gross local errors • long term drift Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200435

  36. Motion from images: difficulties (5) Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200436

  37. Motion from images: difficulties (6) • 151 images, 23 points • manually corrected Lucas-Kanade Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200437

  38. Motion from images: difficulties (7) • squares: ground truth points • dash-dotted line: accurate estimate • solid line: image-only, bundle adjustment estimate Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200438

  39. Outline Motion from images Motion from image and inertial measurements inertial sensors algorithms and results related work Robust image feature tracking Long-term motion estimation Conclusion Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200439

  40. Motion from image and inertial measurements: inertial sensors (1) • inertial sensors can be integrated to estimate six degree of freedom motion Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200440

  41. Motion from image and inertial measurements: inertial sensors (2) But many applications require small, light, and cheap sensors Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200441

  42. Motion from image and inertial measurements: inertial sensors (3) Integrating the outputs of these low grade sensors will produce drifting motion because of: • noise • unmodeled nonlinearities Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200442

  43. Motion from image and inertial measurements: inertial sensors (4) • And, we can’t even integrate until we can separate the effects of… • rotation ρ • gravity g • acceleration a • slowly changing bias ba • noise n • …in the accelerometer measurements Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200443

  44. Motion from image and inertial measurements: inertial sensors (5) Image and inertial measurements are highly complementary With inertial measurements we can: • decrease sensitivity in image-only estimates • establish two rotation angles without drift • establish the global scale Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200444

  45. Motion from image and inertial measurements: inertial sensors (5) Image and inertial measurements are highly complementary With inertial measurements we can: • decrease sensitivity in image-only estimates • establish two rotation angles without drift • establish the global scale …even with our low-grade sensors Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200445

  46. Motion from image and inertial measurements: inertial sensors (6) With image measurements, we can: • reduce the drift in integrating inertial data • distinguish between… • rotation ρ • gravity g • acceleration a • bias ba • noise n …in accelerometer measurements Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200446

  47. Motion from image and inertial measurements: algorithms and results (1) This work has developed both: • batch • recursive algorithms for motion from image and inertial measurements Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200447

  48. Motion from image and inertial measurements: algorithms and results (2) Gyro measurements: • ω’, ω: measured and actual angular velocity • bω: gyro bias • n: gaussian noise Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200448

  49. Motion from image and inertial measurements: algorithms and results (3) Accelerometer measurements: • ρ: rotation • a’, a: measured and actual acceleration • g: gravity vector • ba: accelerometer bias • n: gaussian noise Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200449

  50. Motion from image and inertial measurements: algorithms and results (4) • batch algorithm minimizes a combined error: Dennis Strelow -- Motion estimation from image and inertial measurements – December 13, 200450

More Related