Superquadric Recovery in Range Images via Region Growing influenced by Boundary Information

Superquadric Recovery in Range Images via Region Growing influenced by Boundary Information Master-Thesis Christian Cea Bastidas

Contents • Motivation and Objectives • Superquadric and Rim • Overview of the Proposed Solution • Superquadric Fitting and Rim Fitting • Proposed and Alternative Solution • Evaluation Methodology and Comparison • Summary Christian Cea Bastidas

Motivation • may be modeled with a low fitting error, using a type of surface called Superquadric • are not covered in the image by another object To solve the Segmentation and Recovery Problem which consists in extracting from a 3D image the objects that: Christian Cea Bastidas

To develop an algorithm which solves the stated problem by completing the solution of the existing algorithm Seed Generation. To compare the improved solution with that of the well known approach Recover-and-Select Segmentation ( Leonardis, 1990 ). Objectives Christian Cea Bastidas

Superquadric : Modeling Element Superquadrics, a generalization of the quadric, were chosen as Modeling Object because: • They possess a simple mathematical formulation • The presence of superquadric-like objects is recurrent in many applications. • Its representation capacity can be easily incremented by means of Deformations. Christian Cea Bastidas

Superquadric : DefinitionParametric representation Observation : Superellipsoids, a special type of Superquadric has been considered, which are closed and connected. Christian Cea Bastidas

Superquadric : Examples (I) The number of edges increases as distance themselves from 1. Christian Cea Bastidas

Superquadric : Transformations • Euclidean Transform 6 new parameters => Superquadric needs 11 parameters • Global Deformations ( Bending and Tapering ) Christian Cea Bastidas

Superquadric : Examples (II) A cylinder along its circular and parabolic deformations Christian Cea Bastidas

Rim : Definition The 3D points in a range image are collected by a laser sensor located on a certain plane. The normal to this plane corresponds to the Viewing Point. Assumption : The distance between the laser sensor and the objects is supposed to be large Christian Cea Bastidas

Rim : Example (I) The rims have been drawn for the objects in the image.Viewing point is (0,1,0) ( Axis Y ) Christian Cea Bastidas

Rim : Superquadric Rim A parametric representation of the rim is derived from a more operative definition : => Rim equation Important Property : It permits to sample the rim efficiently ! Christian Cea Bastidas

Rim : Example (II) Rim of a superquadric in general position Viewing Point = (0,1,0) Christian Cea Bastidas

Solution Part 1 : Seed Generation Seed Generation+Edge Detection Input : Range Image Output : - Seeds - Edge Map Seed : Set of points which belong with high probability to a single objectEdge Map : Points on the rims and edges ( All sets areUndistinguishable ! ) Christian Cea Bastidas

Solution Part 2 : Region Growing(1) Region Growing Output : Recovered Objects Input : - Seeds - Edge Map Key Idea : Alternate fitting of the superquadrics to the range image with the fitting of the superquadric rims to the edge map. Christian Cea Bastidas

Solution Part 2 : Region Growing(2) O ( A Seed ) Superquadric Fitting Superquadric is fitted to O Rim is fitted to the Edge Map using the rim of as first estimate Rim Fitting O = O* O* : set of points in the range image whose projection on the plane XZ is inside the Rim Projection Rim Filter No Stop? If O*~ O or Big Error Fitting the Stop! Yes O* ( Recovered Object ) Christian Cea Bastidas

SQ Fitting : The Problem (I) The problem can be stated as follows: Restriction : The points in S come from the visible part of the object ( Self-Occlusion ) Christian Cea Bastidas

SQ Fitting : The Problem (II) Minimization Problem : Preliminary Formulation Self-Occlusion → Christian Cea Bastidas

SQ Fitting : Objective Functions 3 Alternatives for the function :Standard Euclidean Distance (SED) - There does not exist closed mathematical formula - S does not contain necessarily the closest point to an arbitrary point because of the Self-Occlusion. ← UnfeasibleRadial Euclidean Distance (RED) - It has a closed mathematical formula - A good approximation to SED.Modified Algebraic Distance (MED) ← Selected - Closed mathematical formula and simple derivatives - Broadly used in the literature Christian Cea Bastidas

SQ Fitting : SED and RED But SED and RED are more intuitive error measures SED RED Christian Cea Bastidas

SQ Fitting : The Problem (III) Using a modified algebraic distance for : where Definitive Formulation (Solina and Bajcsy ) and reverse the effect of deformations and euclidean transformation respectively. Christian Cea Bastidas

SQ Fitting : Type of Problem The formulated problem : Corresponds to a Nonlinear Least Squares Minimization: For this kind of problem, Levenberg-Marquardt Algorithm is specially suitable. Christian Cea Bastidas

SQ Fitting : LM Algorithm Iterative Procedure defined by : holds for Nonlinear Least Squares Minimization Requisites : • The initial estimate => An Ellipsoide ( Rosenfeld and Kak ) • The derivatives in order to evaluate and Christian Cea Bastidas

SQ Fitting : Examples • The original points ( in pink ) lies on the visible part of the object- Rounded objects are more easily fitted than objects with edges. Christian Cea Bastidas

Rim Fitting : The Problem (I) There exist 2 mayor differences respect to the SQ Fitting : • The real rim of an object cannot be isolated from the Edge Map => Objective Function = Sum of the distances from each point in a sampling of the SQ Rim to the Edge Map. Important Assumptions : - Edge Map contains enough points for each rim - The points on the rim sampling must be uniformly distributed. Christian Cea Bastidas

Rim Fitting : The Problem (I) There exist 2 mayor differences respect to the SQ Fitting : • The real rim of a object cannot be isolated from the Edge Map => Objective Function = Sum of the distances from each point in a sampling of the SQ Rim to the Edge Map. Important Assumptions : - Edge Map contains enough points for each rim - The points on the rim sampling must be uniformly distributed. • The fitting is done in 2D: Rim Sampling and Edge Map are projected onto the plane XZ before computing the distances. Reasons : - Efficiency - Rim Filter needs only the Rim Projection Christian Cea Bastidas

Rim Fitting : The Problem (II) Mathematical Formulation Christian Cea Bastidas

Rim Fitting : Examples Case 2 Case 1 The estimate comes from a Superquadric fitting a small region => stays far from the real Rim The estimate comes from a Superquadric fitting a big region => evolves nearly into the real Rim Christian Cea Bastidas

Region Growing : Algorithm (1) Stop Criterion Parameters Christian Cea Bastidas

Region Growing : Algorithm (2) Output Christian Cea Bastidas

Recover-and-Select Segmentation:Part 1: Seed Generation and Expansion Seed Generation+Region Expansion • Partition the image into nxn regions • Fit a superquadric to each region • Add new points to a region if they are well approximated by the associated superquadric • Go to 2. until no more suitable points are available Christian Cea Bastidas

Recover-and-Select Segmentation:Part 2: Model Selection Model Selection A subset M’ of the generated models M is selected by solving a Binary Quadratic Programming Problem : m : decisionbinary vector The idea is to minimize the information quantityI needed to represent the image : Information I = Bits for SQ parameters + Bits for Error Fitting + Bits for Free Points Christian Cea Bastidas

Evaluation Methodology : Reference Solution The Ideal Solution has 2 parts, one related to the Segmentation and the second one to the Modelling : • The objects are segmented manually from the image and their points are stored as sets • For each object , the superquadric with the best fitting is found. Thus the set constitutes the second part of the solution. As the best fitting cannot be guarranteed, then the Modelling part is replaced by the from the SQ Fitting. The Segmentation part continues being the ideal one. Christian Cea Bastidas

Evaluation Methodology : Indexes 1. Each object O in the solution is matched manually with an object O* in the reference solution.2. Then the Solution Quality is evaluated in three aspects : Modelling belongs to the solution and is the superquadric fitted to O. Segmentation : Convex hull of the projection of the set onto the plane XZ Time 3. Finally each index is averaged over the objects exposed in the image weighting with the size of each set O ( |O| ) Christian Cea Bastidas

Algorithm Comparison : Images The difficulty in solving the recovery problem for an image depends on : • Number of Objects (No) • Average size of the Objects (Size) [ small, medium, large] • Shape of the Objects (Shape) [rounded, box-like, mixed] • Percentage exposed objects or closely exposed (%E.O.) The algorithms were tested using 8 images with the following characteristics : Christian Cea Bastidas

Algorithm Comparison : Index 1 The superquadrics from Alg 1 model better the objects than Alg 2. The exception is the Image 4. For images 5, 6, 7 and 8 the models of Alg 1 are nearly as good as those of the reference solution. Christian Cea Bastidas

Algorithm Comparison : Image 4 The seed is completely contained in on one face of the box => A seed should always contain points on “key sectors” of an object Christian Cea Bastidas

Algorithm Comparison: Index 2 In general, Alg 1 excels in segmenting, except for image 8. Even for the image 8, this index is still good for Alg 1. Christian Cea Bastidas

Algorithm Comparison : Image 8 The rim did not reach the bottom edges of the object. Christian Cea Bastidas

Algorithm Comparison : Index 4 Average Recovery Time (ART) Algorithm 1 : 35 sec. Algorithm 2 : 300 sec. Only in one case Alg 2 was faster than Alg 1. Christian Cea Bastidas

Algorithm Comparison : Index 4 Image 6 is a important case because : - Complexity of the Image - The seeds are not so big Average Recovery Time = 52 sec. Christian Cea Bastidas

Summary • The proposed solution showed a best performance both in efficacy and in efficiency in comparison with the classical approach. Christian Cea Bastidas

Summary • The proposed solution showed a best performance both in efficacy and in efficiency in comparison with the classical approach. • The objective function used for the SQ fitting could be improved considering algebraic approximations to the Standard Euclidean Distance. Christian Cea Bastidas

Summary • The proposed solution showed a best performance both in efficacy and in efficiency in comparison with the classical approach. • The objective function used for the SQ fitting could be improved considering algebraic approximations to the Standard Euclidean Distance. • The parameterization and sampling of the rim played a key role in the solution. Christian Cea Bastidas

Superquadric Recovery in Range Images via Region Growing influenced by Boundary Information