Mesh segmentation
1 / 69

- PowerPoint PPT Presentation

  • Uploaded on

Mesh Segmentation. Zhenyu Shu 2008.5.21 . References. Gelfand N, Guibas L J. Shape segmentation using local slippage analysis [C]. Proceedings of the 2004 Eurographics/ACM SIGGRAPH symposium on Geometry processing, Nice, France, 2004. Nice, France: ACM, 2004: 214-223.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about '' - kalli

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Mesh segmentation l.jpg

Mesh Segmentation

Zhenyu Shu


References l.jpg

  • Gelfand N, Guibas L J. Shape segmentation using local slippage analysis [C]. Proceedings of the 2004 Eurographics/ACM SIGGRAPH symposium on Geometry processing, Nice, France, 2004. Nice, France: ACM, 2004: 214-223.

  • Katz S, Leifman G, Tal A. Mesh segmentation using feature point and core extraction [J]. The Visual Computer. 2005, 21(8): 649-658.

  • Podolak J, Shilane P, Golovinskiy A, et al. A planar-reflective symmetry transform for 3D shapes [C]. ACM SIGGRAPH 2006 Papers, Boston, Massachusetts, 2006. Boston, Massachusetts: ACM, 2006: 549-559.

  • Reniers D, Telea A. Hierarchical part-type segmentation using voxel-based curve skeletons [J]. The Visual Computer. 2008, 24(6): 383-395.

Mesh segmentation3 l.jpg
Mesh Segmentation

  • Become a key ingredient in many geometric modeling and computer graphics tasks and applications

    • Parameterization

    • Texture mapping

    • Shape matching

    • Morphing

    • Multiresolution modeling

    • Mesh editing

    • Compression

    • Animation

    • And more

Mesh segmentation4 l.jpg
Mesh Segmentation

  • Base Definition:

    Mesh segmentation : Let M be a 3D boundary-mesh, and S the set of mesh elements which is either V, E or F. A segmentation of M is the set of sub-meshes = {M0, M1... ,Mk−1} induced by a partition of S into k disjoint sub-sets.

Mesh segmentation6 l.jpg
Mesh Segmentation

  • The key question in all mesh segmentation problem is how to partition the set S. And this relies heavily on the application.

  • Mesh segmentation as an optimization problem:

    • Given a mesh M and the set of elements S ∈ {V, E, F}, find a disjoint partitioning of S into S0, ..., Sk−1 such that the criterion function J = J(S0, ... , Sk−1) be minimized (or maximized) under a set of constraints C.

Constraints l.jpg

  • Commonly used constraints:

    • Cardinality

      • Bound on the maximum and/or minimum number of elements in each part Sito eliminate too small or too large partitions.

      • Bound on the ratio between the maximum and minimum number of elements in all parts to create more balanced partition.

      • Bound on the maximum or minimum number of segments used to balance the partition.

Constraints8 l.jpg

  • Geometric

    • Maximum/minimum area of sub-mesh.

    • Maximum/minimum length of diameter or perimeter of sub-mesh.

    • More complex constraints such as convexity of either 2D patch or volumetric 3D part.

    • Soft constraints in the form of a bias towards specific shapes.

Constraints9 l.jpg

  • Topological constraints

    • Restriction of each Si to be topologically equivalent to a disk.

    • Restriction of each Si to be a single connected component.

Mesh attributes l.jpg
Mesh Attributes

  • Commonly used attributes

    • Planarity of various forms.

    • Higher degree geometric proxies (spheres, cylinders, cones, quadrics developable surfaces).

    • Difference in normals of vertices or dihedral angles between faces.

    • Curvature.

    • Geodesic distances on the mesh.

    • Slippage.

    • Symmetry.

    • Convexity.

    • Medial axis and shape diameter.

    • Motion characteristics.

Shape segmentation using local slippage analysis l.jpg
Shape segmentation using local slippage analysis

Gelfand, Natasha Guibas, Leonidas J

Computer Graphics Laboratory, Stanford University

Proceedings of the 2004 Eurographics/ACM SIGGRAPH symposium on Geometry processing

Main idea l.jpg
Main idea

  • Decompose the mesh into some simple surfaces, such as spheres, planes, cylinders and surfaces of revolution, so-called kinematic surfaces.

Shape classification through slippable motions l.jpg
Shape classification through slippable motions

  • Slippable motion:

    Given a surface S, we call a rigid motion M a slippable motion of S if the velocity vector of each point xS is tangent to S at x.

  • The surface under slippable motions can be thought of as sliding against itself, without forming any gaps between the moving surface and the original copy. That is, the surface S is invariant under its slippable motions.

Kinematic surfaces l.jpg
Kinematic surfaces

  • Surfaces which are invariant under at least one of the types of rigid motions are known as kinematic surfaces.

    • Rotational

    • Translational

    • Helical motions

      POTTMANN H., WALLNER J.: Computational Line Geometry. Springer Verlag, 2001.

Computing slippable motions l.jpg
Computing slippable motions

x is a point belong to the surface S,

n is the normal at x,

r is a rotation vector around x,y,z axis,

t is translation vector.

Computing slippable motions17 l.jpg
Computing slippable motions

  • Eigenvectors of C whose corresponding eigenvalues are 0 correspond to the slippable motions.

  • In practice, due to noise C is likely to be full rank. In this case, the slippable motions are those eigenvectors of C whose eigenvalues are sufficiently small.

Segmentation into slippable components l.jpg
Segmentation into slippable components

  • Goal: Discover a decomposition of P into P1, P2, … ,Pksuch that each Pi is large, connected and slippable.

  • Algorithm:

    • Initialization: compute similarity score between each pair of adjacent patches.

    • Patch growing: at each step, select the most similar adjacent patches and collapse them into a single patch.

    • Termination: stop when the similarity score of all the pair of patches drop below a threshold.

Similarity score l.jpg
Similarity score

  • Two patches Pi and Pj belong to the same component if

    • Their corresponding covariance matrixes Ci and Cj have the same number of small eigenvalues

    • The corresponding slippage signatures are the same, that is we can express the slippable motions of Pi as a combination of slippable motions of Pj and vice versa.

      Let X1...k and Y1...k be the small eigenvalues of Pi and Pj respectively, we just need to test if each column X1...k can be expressed as a linear combination of columns of Y1...k.

Similarity score21 l.jpg
Similarity Score

  • is the (k+1)st singular value of the combined matrix [X1…kY1…k]

  • F is a Gaussian centered around 0 and map small singular values into high similarity scores.

Mesh segmentation using feature point and core extraction l.jpg
Mesh segmentation using feature point and core extraction

Katz, Sagi Leifman, George Tal, Ayellet

Israel Institute of Technology

The Visual Computer. 2005, 21(8): 649-658

Main idea24 l.jpg
Main idea

  • Produce hierarchical segmentations into meaningful components and

    • Be invariant both to the pose of the model and to different proportions between the model’s components.

    • Produces correct hierarchical segmentations of meshes, both in the coarse levels of the hierarchy and in the fine levels.

    • The boundaries between the segments go along the natural seams of the models.

Algorithm overview l.jpg
Algorithm overview

  • Mesh coarsening: to accelerate the algorithm when executed on the large meshes and decrease the sensitivity of noise.

  • Pose-invariant representation: Multi-dimensional scaling is used to transform the mesh S into a pose-invariant representation SMDS.

  • Feature point detection: feature points are computed on SMDS, and mapped back to S.

  • Core component extraction: the core component is extracted.

  • Mesh segmentation: compute the other segments( exclude the core component), each segment contain at least one feature point.

  • Mesh refinement: map the segmentation back to original, fine-resolution mesh.

Pose invariant representation l.jpg
Pose-invariant representation

  • Multi-dimensional scaling: representing dissimilarities as distances in an m-dimensional Euclidean space. The more dissimilar two items are, the larger the distance between them in this space.

  • Here, we define the dissimilarity between points on the mesh as the geodesic distance between them.

Slide27 l.jpg

  • Two major types

    • Metric MDS:

      preserve the intervals and the ratio between the dissimilarities.

    • Non-metric MDS:

      preserve only the order of the dissimilarities, rather than the exact intervals and ratios.

  • Empirical studies shows that non-metric MDS have better results here.

Slide28 l.jpg

  • MDS attempt to minimize

Mds iteration l.jpg
MDS iteration

  • Initialization:

    • Find initial configuration of the points in m-dimensional MDS space (random) (m=3).

  • Iteration:

    • Compute Euclidean distances dij between the points in the MDS space.

    • Using pool adjacent violators algorithm to find the optimal monotonic function f.

    • Each vertex is re-mapped to a point in the MDS space by minimizing Fs.

    • These points are the input of next iteration.

      Barlow, R., Bartholomew, D., Bremner, J., Brunk, H.: Statistical Inference Under Order Restrictions. Wiley, New York (1972)

Mds example l.jpg
MDS example


Feature point detection l.jpg
Feature point detection

  • Feature points:

    • Should reside on tips of prominent components of a given model Intuitively.

    • Should be invariant to the pose of the model.

Feature point detection32 l.jpg
Feature point detection

  • Definition: vertex v is called feature point if

    • It is a local maximum of the sum of the geodesic distance functional, that is,

    • And it resides on the convex-hull of SMDS.

  • Detection: Compute the convex hull of SMDS and then find vertices of convex hull satisfy equation.


Core extraction and segmentation l.jpg
Core extraction and segmentation

  • The segmentation algorithm:

    • Spherical mirroring of SMDS.

    • Extraction of the core component of S.

    • Extraction of the other components of S.

Spherical mirroring l.jpg
Spherical mirroring

  • Find a bounding sphere and mirror the vertices.

Core component extraction l.jpg
Core component extraction

  • Compute the convex hull of the mirrored vertices.

  • The vertices reside on the convex hull, along with the faces they define on S, are considered the initial core component.

Core component extraction36 l.jpg
Core component extraction

  • If initial core does not separate all the feature points, the core component is extended.

  • Core extension: Iteratively add the neighboring faces of the current core until

    • the current core separate all the feature points or

    • The distance from the core to the closest feature point is reduced by more than a constant factor (0.5)

Extraction of other segments l.jpg
Extraction of other segments

  • Extract other segments from mesh by subtracting the core component.

  • A connected component that contains at least one feature point is a segment of the mesh. Component does not contain any feature point joins the core component.


A planar reflective symmetry transform for 3d shapes l.jpg
A planar-reflective symmetry transform for 3D shapes

Podolak, Joshua; Shilane, Philip; Golovinskiy, Aleksey; Rusinkiewicz, Szymon; Funkhouser, Thomas

Princeton University


Main idea42 l.jpg
Main idea

  • Decompose a mesh such that the faces with each segment have the same distinct symmetries.

Symmetry l.jpg

  • Symmetry is an important feature of almost all shapes.

  • Perfect symmetries

    • Be unstable with added noise or missing data

  • Imperfect symmetries

    • Define symmetry distance to measure imperfect symmetries

Symmetry distance l.jpg
Symmetry distance

  • Symmetry distance :

    • Distance between f and the nearest function that is invariant to that reflection

      f is a scalar-valued function defined over a d-dimensional space of points.

      is a plane reflection.

Planar reflective symmetry transform l.jpg
Planar Reflective Symmetry Transform

  • is a mapping from to a scalar-value, which measures f ’s symmetry with respect to the plane

  • Define

    if f is perfectly symmetric with respect to

    if f is perfectly anti-symmetric with respect to

Visualization of the prst l.jpg
Visualization of the PRST

  • The darkness of each point represents the maximum of PRST values over all planes passing through.

Properties of prst l.jpg
Properties of PRST

  • Dominant points and planes of symmetry match our human intuition of the “center” and “major axes”

  • PRST is not sensitive to noise and varies continuously with deformations.

Computation of prst l.jpg
Computation of PRST

  • KATZ, S., AND TAL, A. 2003. Hierarchical mesh decomposition using fuzzy clustering and cuts. Proceedings of ACM SIGGRAPH 22, 3, 954–961.

    has observed that the nearest symmetric function to f is simply the average of f and


Computation of prst49 l.jpg
Computation of PRST

  • To apply PRST definition, convert the surfaces into a binary occupancy grid.

  • Use Gaussian Euclidean Distance Transform

    to smooth for the existence of small noise and small feature

Discrete computation for surfaces l.jpg
Discrete computation for surfaces

  • Adapt Monte Carlo method to efficiently compute for sparsity.

  • Original method:

  • Monte Carlo method:

Weighting l.jpg

  • Consists of two terms:

    • Importance sampling perform, be the reciprocal of the probability of having selected x and x’

    • Change-of-variables weight

Monte carlo estimator l.jpg
Monte Carlo estimator

  • Monte Carlo estimator is

Segmentation l.jpg

  • Find the significant local maxima of the PRST

  • Compute, for each face and every symmetry plane, the degree to which the face contributes to the symmetry with respect to that plane

  • If there are m local maxima in the PRST, then every point has m values representing its support for symmetry with respect to each of the m planes.

Segmentation54 l.jpg

  • Treat these m values as a feature vector and cluster faces according to their proximity in the m-dimensional feature space.

  • For each split, perform k-means( k=2) clustering and take 2 largest connected components and find extra boundary between them using min-cut.

Segmentation55 l.jpg

  • Segmentation terminated condition

    • at a user-supplied depth

    • or when the only planes of local maxima reflect either more than 90% or less than 10% of the surface onto itself.

Hierarchical part type segmentation using voxel based curve skeletons l.jpg
Hierarchical part-type segmentation using voxel-based curve skeletons

Reniers, Dennie; Telea, Alexandru

Eindhoven University of Technology, The Netherlands

The Visual Computer. 2008, 24(6): 383-395

Main ideas l.jpg
Main ideas skeletons

  • Use curve skeleton to segment a voxelized shape

Surface skeleton l.jpg
Surface skeleton skeletons

  • Given a 3D object , the surface skeleton is defined as:

    • a, b are called feature points of p

    • the surface skeleton consists of 2D manifolds is called sheets.

Curve skeleton l.jpg
Curve skeleton skeletons

  • When curve skeletonis not incident with a sheet-intersection curve, define curve skeletonC as:

    • is two shortest geodesics between two feature points of p.

  • When curve skeleton is incident with a sheet-intersection curve, the definition above can not detect correct point.

Curve skeleton61 l.jpg
Curve skeleton skeletons

  • Define the combination of the shortest geodesics for a surface skeleton point p the shortest-geodesic set :

  • Definition of curve skeleton:

  • Junction point:

Component sets l.jpg
Component sets skeletons

  • divide the object surface into multiple components, called component sets

Hierarchical segmentation l.jpg
Hierarchical Segmentation skeletons

  • Let F be the set of foreground components of all critical points combined.

  • Consider all components of F in ascending order of area. For ,

    • if f does not overlap any existing segments in S, add f directly.

    • Else, the potential component s is computed as the set difference between f and the existing segments in S. If the difference add at least 10% of the area it overlaps, add s to S.

Results66 l.jpg
Results skeletons

Advantages l.jpg
Advantages skeletons

  • The borders exhibit minimal twist on the surface and look natural

  • Using geodesics for segment borders yields stable and robust segments for very noisy shapes

  • The segmentations respect the object’s circular symmetry and are invariant for different poses of the same object

Limitations l.jpg
Limitations skeletons

  • Segment borders do not always tightly wrap around attached object parts.

  • The reason is the associated junctions lie deep within the palm, so that the feature points and the ends of

    the geodesics are far

    from the attachment.

Slide69 l.jpg

Thanks skeletons