html5-img
1 / 45

Computer Vision Class project proposal

Computer Vision Class project proposal. Brian Clipp COMP 256. Kalman Filter Implementation of Bundle Adjustment Brian Clipp. Motivation Generate most acurate possible estimate of 3D points and camera poses given multiple views of a scene. Objective.

Jimmy
Download Presentation

Computer Vision Class project proposal

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Computer VisionClass project proposal Brian Clipp COMP 256

  2. Kalman Filter Implementation of Bundle AdjustmentBrian Clipp • Motivation • Generate most acurate possible estimate of 3D points and camera poses given multiple views of a scene. • Objective

  3. Kalman Filter Implementation of Bundle AdjustmentBrian Clipp

  4. Kalman Filter Implementation of Bundle AdjustmentBrian Clipp • Advantages • Variable bundle adjustment window size • Each new P matrix or point added to the bundle adjustment window does not force re-computation of the entire bundle. • Future P matrices are influenced by P matrices no longer in the window, forcing continuity in the model.

  5. Plane Histograms for Planesweeping Stereo David Gallup

  6. Planesweeping Stereo • Hypothesize a plane • Project images from all cameras onto it • Measure photoconsistency (cost) as seen from a reference view • Repeat for a family of planes • Per pixel, record plane with best cost near far Multiway Planesweep • Sweep planes in multiple directions • Better alignment with surface produces better results • Surface normals known through heuristics or other information

  7. Cost Function and Plane Histogram Cost function • Each pixel has a cost function • Chosen depth is plane of minimum cost • Poorly defined minimum in some cases • Multiple local minima in some cases Histogram • Histogram 3D points against planes in each direction • Helps to disambiguate poorly defined minima • Provides temporal coherency

  8. Cost Function and Plane Histogram

  9. Occlusion Inference with Visual Hull Li Guan

  10. Probabilistic Occupancy Grid • Jean-Sebastien Franco, “Fusion of Multi-View Silhouette Cues Using a Space Occupancy Grid”, ICCV, 2005 • See the Video

  11. Project Goal • To formalize the inference of POccluder from POccupancy Grid For every view: (1-POccluder)* Pd=POccupancy Grid • To infer occlusion masks for a single view • Similar to the occupancy grid, to have an occlusion grid for the environment

  12. Continuous Calibration Tyler Johnson COMP 256 Spring ‘06

  13. Calibration in Tiled Displays • Multiple overlapping projectors must be calibrated • Large up-front calibration process • Stereo cameras triangulate display surface and calibrate projectors using correspondences

  14. Continuous Calibration • Would like to be able to continuously refine estimate display surface and projector parameters • Structured light vs. feature detection in continuous image streams • Current structured light methods sacrifice image quality -> feature detection/correlation • Requires dynamic scene with available features

  15. Proposal • Initial calibration of projector, stereo camera pair and estimate of surface geometry • Detect features in the user imagery observed by camera and track with optical flow • Given calibration, find features in other views • Use matched features to re-estimate disp. surface/projector calibration • Kalman filter aids in estimation and provides level of uncertainty • Predictor-Corrector

  16. 3D Hole Filling Using Texture Synthesis Concepts COMP 256 John Mason

  17. 3D Hole Filling Using Texture Synthesis Concepts John Mason • 3D geometry • 3D by hand • High quality • Takes time and people • 3D from scans • Tends to have holes where scanner can’t reach or due to other surface qualities • Humans easily distracted by unexpected holes • Multiple scans can’t always fill the holes • How to fill the holes? Images: Davis & Levoy

  18. 3D Hole Filling Using Texture Synthesis Concepts John Mason • Texture Synthesis • Take a small piece of texture • Make a bigger piece of texture • Fill in gaps in texture • 3D Hole Filling • Look at hole’s neighborhood • Determine local geometric features • Create a reasonable representation of the missing geometry • Looks like it belongs or exact match? Image: Wei & Levoy Image: Wei & Levoy Images: Efros & Leung Images: Davis & Levoy

  19. 3D Hole Filling Using Texture Synthesis Concepts John Mason • Project Target • Explore value of various texture synthesis methods applied to 3D hole filling • Develop rapid, automated hole filling implementation • Make sure filled hole looks like it should belong there • As time allows, compare to other hole filling technologies Prior work in the area of Texture Synthesis, and related disciplines, includes works by Efros & Leung, Efros & Freeman, Turk & Levoy, Wei & Levoy, Kwatra, and Lefebvre & Hoppe among others. There is additional work recreating 3D textures and surfaces including works by Dong & Chantler, Torrez & Dudek, Ricken & Warren, Velho et al, and Davis & Levoy among others.

  20. Paul Merrell Project Proposal COMP 256

  21. Hole Removal • 3D Reconstructions from stereo often have holes. • The holes are a result of occlusions. • Goal of my project is to fill in the holes. • Doesn’t need to be perfect, just better than the holes.

  22. Scene with an Occlusion Use Smoothness to fill in hole w/o texture Texture Synthesis to Create Texture

  23. 2D Hole Filling Using Texture Synthesis 3D Hole Filling Copy from Similar Parts

  24. Parametric (Global) Optic Flow for Stabilization Brad Moore

  25. Goal • Evaluate and implement parametric (global) optical flow • Speed of algorithm • Points of failure

  26. Goal (cont.) • Apply to the problem of stabilization (removing jitters) • Implement a real-time demo

  27. Approach • Development on Windows PC • Program in Matlab • Input Device: webcam (purchase, acquire) or video from digital camera • Possible Issues • Speed • Noise (quality of camera)

  28. Ultrasound Calibration Tabitha Peck

  29. Ultrasound Calibration Tabitha Peck • Problem: How do you determine the relationship between the location of an ultrasound probe and the real world location of the objects in the image?

  30. The Setup Tabitha Peck Ultrasound Probe Optotrak Certus Tracker Image Phantom

  31. Challenges Tabitha Peck • Distinguishing items within the phantom • You do not know the exact location of items in the phantom • Error produced by Optotrak Certus 0.1 mm for x, y coordinates 0.15 mm for z coordinate • Ultrasound images are not 2-D

  32. Image Segmentation for Display Surface Reconstruction • Overview (WAV): • The goal - Robust automatic calibration of projectors displaying on complex surfaces, using commodity hardware (i.e. cameras, projectors, & PCs) R. Skarbez -- 20 Feb 2006

  33. Image Segmentation for Display Surface Reconstruction • One aspect of this problem is the accurate modeling of the surface geometry • WAV has used several methods: • Tessellation of the 3D point cloud • Benefits: • Extremely general (Makes no assumptions about the underlying display geometry) • Drawbacks: • Very inefficient (1000s-100000s of polys) • Noisy (Small errors make surface look “wavy”) • Does not reconstruct corners well (BIG problem) R. Skarbez -- 20 Feb 2006

  34. Image Segmentation for Display Surface Reconstruction • Previous attempts, cont’d: • Fitting planes to data (Paper in EDT ’06) • Benefits: • Many fewer polygons (1s-10s) • Much less noise (Surface forced to be planar) • More accurate corner detection than previously • Drawbacks: • Recursively fit on entire point cloud – can generate spurious or redundant planes • Corners still not perfect – dependent on quality of fitted planes • Only works on piecewise-planar display surfaces • Want to get best of all worlds – Generality, efficiency, low noise, and better corner detection • How? Segment the image, and process each segment independently R. Skarbez -- 20 Feb 2006

  35. Image Segmentation for Display Surface Reconstruction • Method • Scan lines across the display surface (using the projectors) • Segment the camera images based on discontinuities in the observed lines • Process each segment individually (e.g. fit a plane if possible, if that fails, try fitting a quadric, or tessellate the data) • Reduces to plane-fitting in the simple case (albeit without spurious planes, and possibly with more accurate corners, due to the fact that they are directly located) • In the general case, allows for handling of arbitrary display surfaces at the cost of generating a more complex display model R. Skarbez -- 20 Feb 2006

  36. Avatars in VR • Jeremy Wendt • Avatars - representation of the user in the environment • Improve Presence • Requires tracking several joint locations

  37. Camera Tracking • Needs to be fast • 30 fps • latency below 100ms • Needs to be robust • Jerky motion or unpredictable update break presence • Large tracked area, low cost would be benefits… not necessary

  38. Proposed System • Use brightly colored markers • Required (Real Time+UI): • 0. Calibrate cameras • 1. Initialize color markers for cameras • 2. Track the color blob center images • 3. Find 3 space coordinate for tracked position • Bonus: • 4. Smoothing results between frames • 5. Placing avatar at specified location • 6. Use in HMD tracked space

  39. Aerial Image registration- Changchang Wu • Image sources • satellites (USGS) • helicopters • GIS (Google Earth,…) • 3d model screenshot • The matching problem • Matching of different image sets • Matching the images with 3d model • Images with elevation information • Retrieve the geographical location

  40. Image Matching • Local matching of SIFT features • Scale Invariant Feature Transform (Lowe99) • Select the best match for each feature • Look for local constraints for generating reliable local matches in this particular problem • Look for strategies of selecting more discriminative SIFT features • Global matching • Improve the efficiency and robustness of the current matching algorithm • Implement the matching of images with elevation data, use RANSAC to fit the model.

  41. 667 putative matches 34 correct matches A matching example • 6032 SIFT in the left, and 5591 in the right ( part of them are displayed)

  42. Normal Recovery from Reflective Surfaces Talha Zaman COMP 256

  43. The Idea • Urban environments often have buildings with large reflective surfaces (plate glass windows) • The reflections from the surfaces contain a large amount of information about the surface geometry • The goal is to use this information to recover the normals and reconstruct the shape of the reflective surface

  44. The Approach • Take a picture (or several pictures) of the reflective surface • Take pictures of the reflected scene directly (from same camera position?) • Figure out the relationships between the surface, the camera, and the scene • Relate reflected image to real image and recover normals • Reconstruct the surface

More Related