Recognition I: Extended Gaussian Images

1. Recognition I:Extended Gaussian Images Andrew Nashel COMP 290-075: Computer Vision http://www.cs.unc.edu/~nashel/290-075/

2. Overview • Motivation • Gaussian Image • Extended Gaussian Image • Gaussian Curvature • Extended Circular Image • Complex EGI

3. Motivation • Object recognition is often one of the ultimate goals for vision systems. • It is necessary for real world interactions such as: • Navigation through environments • Robotic handling of objects • Object inspection • It brings together many components of computer vision: • Depth extraction • Image segmentation • Geometric modeling

4. ˆ ˆ n n ~ ~ The Gaussian Image • Surface normal information for any object may be mapped onto a unit (Gaussian) sphere by finding the point on the sphere with the same surface normal:

5. Properties of the Gaussian Image • This mapping is called the Gaussian image of the object when the surface normals for each point on the object are placed such that: • tails lie at the center of the Gaussian sphere • heads lie on the sphere at the matching normal point • In areas of convex objects with positive curvature, no two points will have the same normal. • Patches on the surface with zero curvature (lines or areas) may correspond to a single point on the sphere. • Rotations of the object correspond to rotations of the sphere.

6. The Extended Gaussian Image • We can extend the Gaussian image by • placing a mass at each point on the sphere equal to the area of the surface having the given normal • masses are represented by vectors parallel to the normals, with length equal to the mass • An example: EGI of Block Block

7. Using the EGI • EGIs for different objects or object types may be computed and stored in a model database as a surface normal vector histogram. • Given a depth image, surface normals may be extracted by plane fitting. • By comparing EGI histogram of the extracted normals and those in the database, the identity and orientation of the object may be found.

8. d d b b e e a a Problems with the EGI • EGIs only uniquely define convex objects. • An infinite number of non-convex objects may have the same EGI: Areas: a+b=c d+e=f e c

9. Gaussian Curvature • Formally, we will develop the extended Gaussian image based upon the Gaussian curvature of the object. • Consider a patch of area O on the object, and the corresponding area S on the Gaussian sphere:

10. S dS K = lim = O dO O0 Defining Gaussian Curvature • Given patches O and S, we define Gaussian curvature K as the limit of the ratio of the two areas as they approach zero: • If the object surface is strongly curved, then the corresponding points on the Gaussian sphere will be spread out. • If the surface is planar, the normals will be parallel and will map to a single point on the Gaussian sphere.

11. Defining Curvature Continued • If we integrate over a patch O on the object we have: where S is the area of the patch on the Gaussian sphere. • We call the expression on the left the integral curvature. This allows us to handle surfaces with discontinuities in surface normals. ∫∫O K dO = ∫∫SdS = S

12. Defining Curvature Continued • Similarly, if we integrate over a patch S: where O is the area of the patch on the object. • This relationship suggests that the inverse of the curvature will be used to define the extended Gaussian image. ∫∫S 1/K dS = ∫∫OdO = O

13. 1 G(,) = K(u,v) Defining EGI • Let u and v be used to specify points on the original surface, and let  and  specify points on the Gaussian sphere. • We now define the extended Gaussian image as: the inverse of the Gaussian curvature, where (,) is the point on the Gaussian sphere corresponding to the point(u,v) on the object.

14. The Discrete Case EGI • To represent the information of the Gaussian sphere in a computer, the sphere is divided into cells: • For each image cell on the left, a surface orientation is found and accumulated in the corresponding cell of the sphere.

15. Discrete Approximation • In an actual implementation of a discrete EGI, we start with a surface orientation map. • Shown here is a needle diagram of an inclined torus obtained by photometric stereo:

16. Orientation Histogram • The discrete approximation of the EGI is called the orientation histogram. • The needle diagram of the torus is projected onto a tessellated unit sphere to create an orientation histogram, displayed as a set of spikes:

17. The Extended Circular Image • The extended circular image is the 2-D equivalent of the extended Gaussian image.

18. Polygon Morphing with the ECI • An alternative to pixel-based morphing algorithms for convex polygons: • First compute the ECI representation of the source and target polygons. • Match source and target normals on the ECI circle to create source-target pairs. • Interpolate weights and angles between pairs to find the ECI of intermediate steps. • Reconstruct the convex polygon from the ECI. • Java implementation: http://web.mit.edu/manoli/ecimorph/www/code/MMorph.html

19. The Complex EGI • Another problem with the EGI is that the weights in the representation only encode area information and not positional data, thus it is impossible to determine translation. • The Complex EGI is an alternative formulation in which the weight at each discrete cell is a complex number: • The magnitude at each cell is the surface area as in the standard EGI. • The phase is the signed distance of the surface patch from a designated origin along the normal.

20. The Complex EGI • We see that it is a simple modification to handle displacement determination:

21. Conclusions • The extended Gaussian image is a useful technique for representing the shape of an object, and even its position (complex EGI). • However, it is only a component tool to be used in a shape recognition process which also includes: • Surface orientation determination • Image segmentation into objects • Prototypical models/object databases • System control - what object to handle/inspect?

22. References • Horn, B.K.P. 1984. Extended Gaussian images. In Proceedings of the IEEE 72, 12 (Dec.), pp. 1656-1678. • Horn, B.K.P. 1986. Robot Vision. MIT Press, Cambridge, MA, pp. 365-399. • Kamvysselis, M. 1997. 2D Polygon Morphing using the Extended Gaussian Image. http://web.mit.edu/manoli/ecimorph/www/ecimorph.html • Kang, S.B. and K. Ikeuchi. 1990. 3-D Object Pose Determination Using Complex EGI. tech. report CMU-RI-TR-90-18, Robotics Institute, Carnegie Mellon University.