3D Skeletons Using Graphics Hardware

1. 3D Skeletons Using Graphics Hardware Jonathan Bilodeau Chris Niski

2. Outline • Background • Goal • Method • Find points that lie on the graph • Extract vertices/edges from points • Results • Problems • Future Work

3. Background • Two general techniques for computing shape descriptors • Statistical shape descriptors • Structural shape descriptors • Statistical shape descriptors work well when comparing objects that cannot be deformed • Also need to potentially store large amounts of data

4. Background • Structural shape descriptors are simple geometrical descriptions of a more complex model • Generally the structural shape descriptor takes the form of a skeleton of the mesh • Skeleton representation is efficient if properly computed as it takes up little space, and can compare deformed objects, as well as perform partial matching

5. Computing the skeleton • In 2D the traditional method of computing the skeleton is to use the Euclidian Distance Transform • Basically find the center line of a 2d contour • Another method is to use waves propagating inward from the surface

6. Example borrowed from Mishas class notes Surface EDT

7. The EDT is expensive to compute • On2m for 2D • On3m for 3D • No good/fast 3D equivalent • Usually mesh thinning is used to calculate the skeleton

8. Goal • Using graphics hardware, create a 3D graph of a model • Slice the model along each coordinate axis • Find 2D skeleton with GH, merge them • Find intersection • Extract graph • Capture 3d axis information • Intersection will remove random noise • Simple transition to a graph • only vertices and edges • No sheets

9. Method • Voxelize the model • Consider each slice of the volume along each axis • Gives you a 2D contour • Compute the skeleton of each 2D contour

10. Method – Finding 2D skeleton • Using Graphics hardware • For each point on the contour draw a code centered on the point, pointing at the viewer. • Read back the depth buffer • Analogous to a distance transform

11. Method – Finding 2D skeleton • Look for discontinuities in the EDT • Scan columns/rows looking for points where both neighbors are smaller • Every point that passes is marked true

12. Method • Output is a 2D skeleton • Merge the 2D skeletons into a volume again • Intersect each volume • One from each axis

13. Method – Noise • Noise around the perimeter

14. Method – Noise • Red dots indicate points that allowed a skeleton point in the center of this circle

15. Method – Noise Removal • Angle between blue lines is small • Remove points that have a small angle

16. Method – Noise Removal • Keep points above 90 degrees • D < Sqrt(2)*R

17. Method • Once the EDT of the mesh is calculated we need to fit a skeleton to the resulting points • The EDT voxel grid is not necessarily connected, ie. Voxels may not have neighbours

18. Fitting Skeleton • Start by assigning an edge to each point that has several neighbours • Begin merging the edges based on several criteria • Distance between the two edges • Length of the edges • The angle between the two edges • Stop merging when no suitable candidates are present

19. Fitting Skeleton • We also discard some of the edges • If they are short and do not have an edge to merge with • Not attached to the rest of the graph

23. Results • We can extract a voxel grid containing the mesh skeleton • Filter the grid to remove some of the noise • Fit a skeleton onto the grid to compactly represent the grid points

24. Sample results

25. Problems - Voxelation • Still can’t remove all the noise • Have to be conservative with the noise test or else you throw out large chunks of the valid skeleton

26. Problems - Orientation • Objects that aren’t axis aligned don’t work. • Results in sheets • Traditional 3D MA has sheets • Contradicts one of our goals

27. Problems • General problems with skeleton based approach: • The skeleton is not very descriptive • Can be very susceptible to noise • If connectivity changes after deformation, comparison becomes very difficult • Hard to capture just the right amount of detail

28. Problems with our approach: • Our version of the EDT still creates noise around the surface of the mesh • Noise on the surface of the mesh creates false edges which can merge with good edges

31. Future Work • Try other methods for finding 2D skeleton that are less prone to noise • Before voxelation? • High resolution 2D images that get down sampled? • Can’t keep high resolution for 3D • What affect would aliasing have?

32. Future Work • Orientation problem • Instead of using the coordinate axis, find the EGI with a relatively small number of bins. • Flip any normals with –z components • Find axis volume relative to the dominant normals • Problem lining up volumes with arbitrary rotations