1 / 49

Shapelets Correlated with Surface Normals Produce Surfaces

This research explores a method to calculate the surface normal direction at each point in an image and infer the surface shape from the normals. It addresses the ambiguities in tilt and applies regularization schemes to deal with noise. The use of localized, non-orthogonal basis functions and correlations with slant and tilt provides more accurate reconstructions of surfaces.

dduvall
Download Presentation

Shapelets Correlated with Surface Normals Produce Surfaces

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Shapelets Correlated with Surface Normals Produce Surfaces Peter Kovesi School of Computer Science & Software Engineering The University of Western Australia

  2. Shape from Shading and Shape from Texture attempt to • Calculate the surface normal direction at each point in the image, and then • Infer the surface shape from the surface normals.

  3. N slants Surface normals are usually calculated in terms of slant and tilt eye surface Slant is the angle between the viewing direction and the surface normal

  4. Tilt is the angle of the projection of the surface normal on a plane perpendicular to the viewing direction projection of surface normal tilt t N slant s plane perpendicular to viewing direction viewer

  5. Problem: Tilt often has an ambiguity ofp

  6. Problem: Tilt often has an ambiguity ofp

  7. Problem: Regularization schemes are often used to deal with noise. However, they typically involve the minimization of an Energy Term that has mixed units surface gradient error (highly nonlinear with surface angle) shading error bending energy

  8. Problem: Regularization schemes are often used to deal with noise. However, they typically involve the minimization of an Energy Term that has mixed units surface gradient error (highly nonlinear with surface angle) shading error bending energy • The weighting parameters have to act as unit conversion operators. • The appropriate values will be dependent on scale. • This can be problematic if the regularization is being applied within a multi-scale framework. • Discontinuities have to be automatically detected and treated specially.

  9. Alternative approach: Project the surface gradients onto a set of integrable basis functions and reconstruct the surface from these. Frankot and Chellappa [1988] first introduced this approach using Fourier basis functions. • Features of work presented here: • Localized, non-orthogonal basis functions are used. • Correlations with basis functions are performed with respect to slant and tilt rather than in terms of gradient in x and y. • Ambiguities of  in tilt can be considered. • Low quality reconstructions are possible with slant information alone.

  10. Decomposing and Reconstructing Signals with Basis Functions signal band-pass filters/wavelets filtered at scale 2 filtered at scale 4 filtered at scale 6 Reconstruction obtained by summing band-passed signals over 6 scales

  11. We only have the gradient of the surface…

  12. We only have the gradient of the surface… Differentiation is linear surface gradient surface band-pass filter

  13. We only have the gradient of the surface… Differentiation is linear A band-passed version of the signal can be obtained from a correlation performed between the gradient of the signal and integral of the band-pass filter. surface gradient Integral of band-pass filter (shapelet gradient) surface band-pass filter

  14. Choice of Shapelet • The gradient of shapelet function must satisfy the admissibility condition of zero mean. • Must ensure preservation of phase information in the signal. • The bank of shapelet filters should provide a uniform coverage of the signal spectrum so that it is faithfully reconstructed. These requirements are satisfied by a Gaussian

  15. Note that correlating the gradient of one shapelet filter with the gradient of the signal will correspond to extracting a band of frequencies from the signal gradient.

  16. Note that correlating the gradient of one shapelet filter with the gradient of the signal will correspond to extracting a band of frequencies from the signal gradient. The signal gradient spectrum differs from the original signal spectrum in that its amplitudes are scaled by the frequency.

  17. Note that correlating the gradient of one shapelet filter with the gradient of the signal will correspond to extracting a band of frequencies from the signal gradient. The signal gradient spectrum differs from the original signal spectrum in that its amplitudes are scaled by the frequency. The sum of the shapelet gradient transfer functions should form a curve that is inversely proportional to frequency to counteract the scaling of the gradient spectrum. The net result will then be an even coverage of the original signal spectrum.

  18. Shapelet bank formed by five Gaussians with height proportional to scale. Transfer functions of the gradients of the shapelets above. The sum forms a curve that is approximately inversely proportional to frequency.

  19. A bank of shapelets will not form an orthogonal basis set This does not matter if we have a strongly redundant basis set. (Achieved by using a continuous wavelet transform.) A non-orthogonal basis set can be considered to be the sum of several orthogonal basis sets. - We get accurate reconstruction up to a scale factor

  20. The scaling of the reconstruction due to the redundancy of the basis set can be determined and corrected if required. • Overcomplete basis sets have the advantage that small changes in local features result in correspondingly small changes in the basis coefficients. • Overcompleteness also provides robustness to noise.

  21. In Summary: • Correlate the gradients/normals of the surface with the gradients/normals of a set of basis functions. • These correlation results correspond to bandpassed versions of the original signal. • Reconstruct surface by summing correlation results. • Perform correlation in terms of slant and tilt separately • - slant and tilt are calculated by different means. • - slant and tilt have differing degrees of uncertainty.

  22. Correlation of Slant Slant lies in the range 0 -p/2which only provides gradient magnitude information. The slant correlation at scale i is formed by With no tilt information this correlation measure matches positive and negative gradients equally. (No distinction between concave or convex shape) gradient of shapelet at scale i gradient of surface

  23. Correlation of Tilt • If gradient magnitudes match and tilts match • correlation is +ve • If gradient magnitudes match and tilts differ by p • correlation is -ve • If tilts differ by p/2 correlation is 0 • Use the cosine of the tilt difference as a correlation measure tilt of shapelet at scale i tilt of surface

  24. Overall Correlation Reconstruction

  25. synthetic test surface needle diagram of surface normals

  26. surface normals reconstruction over 2 scales reconstruction over 4 scales reconstruction over 6 scales

  27. Shapelet reconstruction using the gradients of the range data Range data from OSU database

  28. Slant and tilt with Gaussian noise of 0.3 radians standard deviation Shapelet reconstruction Terzopoulos multigrid reconstruction Frankot Chellappa reconstruction

  29. Allowing for Tilt Ambiguity of p When tilts are aligned value is 1 When tilts are orthogonal value is 0 When tilts are opposite value is 1 … results in rectification of the band-pass components of the surface

  30. Reconstruction with a tilt ambiguity of p bumps surface reconstruction of ramps surface reconstruction

  31. Reconstruction with slant data alone Slant-only reconstruction of bumps Slant-only reconstruction of ramps Sinc function Slant-only reconstruction of sinc

  32. …after segmentation and negation Slant-only reconstruction of face

  33. Real data: Surface normals from texture Shapelet reconstruction

  34. Shapelet reconstruction Sand ridges: Slant set to /4 at all points Tilt set to 0 for white regions,  for black regions

  35. Shape from Occluding Contours Surface normals at occluding contours are perpendicular to viewing direction. This is a powerful cue to shape. Manually marked contours

  36. A crude approximation of the surface normals in the scene… At all points slant is set to 0, except along contours where slant is set to , and tilt is set to be perpendicular to contour. This gradient field is not integrable.

  37. Shapelet reconstruction

  38. Manually drawn contours

  39. Conclusions • Reconstructions from surface normals using shapelets is simple and the process is robust to noise. • The use of basis functions implicitly imposes a continuity constraint in the reconstruction. • Being able to treat slant and tilt separately permits data with tilt ambiguity, or no tilt data, to be considered. • Much can be made from limited amounts of surface normal data. This allows some some aspects of structure to be deduced from line drawings.

More Related