Create Presentation
Download Presentation

Download Presentation

3D Shape Retrieval from Silhouette Images

Download Presentation
## 3D Shape Retrieval from Silhouette Images

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Overview (1)**Input: Output: Edges:**Overview (2)**• Are silhouettes good descriptions of shape? • One may not be… but a few are • Can you see 3D structure in the shadow of a hand in the projector? • Silhouettes are easily entered queries… draw the contour of your shape • Also, they can be derived from an arbitrary sketch or image… or shadows, come to think of it**Approach Overview**• Sea of cameras: produce multiple views of an object • A silhouette contour of a closed object is a circular function: can be made rotation-invariant • Multiple silhouettes can be used to improve the accuracy of a query (i.e. get me the object that has these two silhouettes)**Sea of cameras**• The “sea” of cameras can be sampled over a hemisphere because silhouettes will be the same (mod flip) • Regular distribution on a sphere over samples… • Ideal case: random distribution of K camera points over a hemisphere that changes for each model • My version: custom placed locations of 20 points on a hemisphere that is constant across all shapes**Contour extraction (1)**• Most practical shapes will have: • Sampling artifacts in the silhouette image • Interior contours (i.e. no one dominant contour) silhouette edges**Contour extraction (2)**• Perform morphological closing of silhouette image with a disk structuring element • Use a laplacian-of-gaussian edge detector to extract contour pixels**Contour extraction (3)**• Using a maze walking algorithm, find a closed loop through the contour pixels to produce a single closed group of pixels • This is done for all connected components • I bet we could use all of the components to do partial matching on this shape • … but for now, we chose the largest contour that results**Silhouette Parameterization**• Goals: rotation, scale, translation invariance • RDF (radial distance function) • f(c) = r where r is maximal object extent in direction c from centroid • As in spherical harmonics • Arc length parameterization: • f(c) = r + k(s)i parameterization about centroid • Where k(s) is the stretch required to fit the curve to the surface**Matching Metric**• Let dist(s1, s2) be the sum of squared distances between the descriptor for two contours (“samples”) • Option 1: best sample from query and model • Let dist(mi,q1…K) for some model mi and K samples q1…j be defined as • The sample sd in model mi that is closest to any qi • Option 2: all samples in query, best in model • Sum over all k of qi • The sample sd in model mi that is closest to qi**System Details (1)**• I start with Princeton Shape Benchmark geometry and classifications • Silhouette sea is generated by a glut app in Linux using depth buffer readback • Images are copied to a machine with Matlab where morphological closing and edge detection is done • … and then copied back… grr… • Useful stat: 622 models yields 1.5 GB of images, but only a 7.5mb database**System Details (2)**• Edge images are processed by a maze search algorithm to find the largest closed contour • Contours are sampled evenly to 200 points using Gnu Scientific Library • The parameterization is FFT’d with FFTw • The resulting descriptors are stored in a binary database • We can set an epsilon value that allows the same sample to be appear in multiple models if it is similar enough (compression!) • Precision vs recall is generated internally by the database**Control tests: rotation invariance**• Match is somewhat sensitive to noise… e.g. higher frequency noise introduced by the sampling of the contours**Control tests: rotation invariance**• Tinkering with filtering and other stuff doesn’t help much… using the RDF does.**Circular parameterization andModel match option 1**20 classes shown57 total classes622 models**Circular parameterization andModel match option 1**20 classes shown57 total classes622 models**Circular parameterization andModel match option 1**20 classes shown57 total classes622 models**Radial distance parameterization andModel match option 1**15 classes shown57 total classes622 models 1 silhouette used in query**Radial distance parameterization andModel match option 1**15 classes shown57 total classes622 models 2 silhouettes used in query**Radial distance parameterization andModel match option 1**15 classes shown57 total classes622 models 8 silhouettes used in query**Radial distance parameterization andModel match option 2**1 silhouette used in query 15 classes shown57 total classes622 models**Radial distance parameterization andModel match option 2**4 silhouettes used in query 15 classes shown57 total classes622 models**Radial distance parameterization andModel match option 2**4 silhouettes used in query 15 classes shown57 total classes622 models**Radial distance parameterization andModel match option 2**8 silhouettes used in query15 classes shown57 total classes622 models**Radial distance parameterization andModel match option 2**8 silhouettes used in query, 57 classes shown57 total classes622 models