Shape based retrieval of articulated 3d models using spectral embedding l.jpg
Advertisement
This presentation is the property of its rightful owner.
1 / 24

Shape-Based Retrieval of Articulated 3D Models Using Spectral Embedding PowerPoint PPT Presentation

Shape-Based Retrieval of Articulated 3D Models Using Spectral Embedding Varun Jain and Hao Zhang {vjain,[email protected] GrUVi Lab, School of Computing Science Simon Fraser University, Burnaby, BC Canada Problem Overview … … Problem Overview Database Outline Problem Overview

Related searches for Shape-Based Retrieval of Articulated 3D Models Using Spectral Embedding

Download Presentation

Shape-Based Retrieval of Articulated 3D Models Using Spectral Embedding

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Shape based retrieval of articulated 3d models using spectral embedding l.jpg

Shape-Based Retrieval of Articulated3D Models Using Spectral Embedding

Varun Jain and Hao Zhang

{vjain,[email protected]

GrUVi Lab, School of Computing Science

Simon Fraser University, Burnaby, BC Canada


Problem overview l.jpg

Problem Overview


Problem overview3 l.jpg

Problem Overview

Database

  • Outline

  • Problem Overview

    • Retrieval Problem

    • Methods

  • Spectral Embeddings

  • Shape Descriptors

  • Results

  • Future Work

  • Acknowledgements

Shape Retrieval

Applications

  • Computer aided design

  • Game design

  • Shape recognition

  • Face recognition

Query

Interface

User

Output


Shape retrieval l.jpg

Shape Retrieval

  • Outline

  • Problem Overview

    • Retrieval Problem

    • Methods

  • Spectral Embeddings

  • Shape Descriptors

  • Results

  • Future Work

  • Acknowledgements

How?

  • Using Correspondence

    Efficiency??

  • Using Global Descriptors

    For 3D shapes

    • Fourier Descriptors

    • Light Field

    • Spherical Harmonics

    • Skeletal Graph Matching

  • Non-rigid transformations??

  • Stretching

  • Articulation (bending)


Shape retrieval5 l.jpg

Shape Retrieval

  • Outline

  • Problem Overview

    • Retrieval Problem

    • Methods

  • Spectral Embeddings

  • Shape Descriptors

  • Results

  • Future Work

  • Acknowledgements

Our Method

  • Normalize non-rigid transformations

    • Construct affinity matrix

    • Spectral embedding

  • Use global shape descriptors

    • Light Field Descriptor (LFD)

    • Spherical Harmonics Descriptor (SHD)

    • Eigenvalues??


Shape retrieval6 l.jpg

Shape Retrieval

  • Outline

  • Problem Overview

    • Retrieval Problem

    • Methods

  • Spectral Embeddings

  • Shape Descriptors

  • Results

  • Future Work

  • Acknowledgements

Advantages of Our Method

  • Handles shape articulation (best performance for articulated shapes).

  • Flexibility of affinity matrices

  • Robustness of affinity matrices


Spectral embeddings l.jpg

Spectral Embeddings


Spectral embeddings8 l.jpg

2

(

)

d

i

j

n

g

e

o

;

2

a

e

2

=

¾

i

j

a

a

a

1

1

1

2

1

¢

¢

¢

n

z

}

|

{

a

8

2

3

2

1

¢

¢

¢

>

.

.

.

>

.

.

.

>

>

6

7

.

.

.

A

=

>

n

>

6

7

>

a

a

a

>

6

7

n

n

<

i

i

i

1

2

6

7

¢

¢

¢

n

£

.

.

.

6

7

.

.

.

6

7

.

.

.

>

>

6

7

>

a

a

a

>

6

7

>

>

4

5

1

2

¢

¢

¢

n

n

n

n

>

>

:

Spectral Embeddings

Affinity matrix

  • Outline

  • Problem Overview

  • Spectral Embeddings

    • Basics

    • Problems

    • Solutions

  • Shape Descriptors

  • Results

  • Future Work

  • Acknowledgements


Spectral embeddings9 l.jpg

¯

¯

¯

T

A

E

¤

E

¯

¯

¯

=

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

k

¯

¯

¯

¯

¯

¯

k-dimensional spectral embedding coordinates of ith point of P

z

}

|

{

~

~

~

¯

¯

¯

u

u

¢

¢

¢

u

k

1

2

¯

¯

¯

8

2

3

¯

¯

¯

¯

¯

¯

>

>

>

>

6

7

>

>

6

7

>

.

.

.

>

6

7

>

.

.

.

>

6

7

.

.

.

1

<

6

7

U

E

¤

2

¢

¢

¢

u

u

u

n

=

=

k

k

k

i

i

i

1

2

6

7

k

6

7

>

>

6

.

.

.

7

>

.

.

.

>

6

7

.

.

.

>

>

6

7

>

>

4

5

>

>

:

Spectral Embeddings

Eigenvalue decomposition:

Scaled eigenvectors:

  • Outline

  • Problem Overview

  • Spectral Embeddings

    • Basics

    • Problems

    • Solutions

  • Shape Descriptors

  • Results

  • Future Work

  • Acknowledgements

Jain, V., Zhang, H.: Robust 3D Shape Correspondence in the Spectral Domain. Proc. Shape Modeling International 2006.


Spectral embeddings10 l.jpg

~

~

~

u

u

u

2

3

4

8

2

3

¯

¯

>

¯

¯

>

>

>

6

7

¯

¯

>

>

6

7

>

¯

¯

.

.

.

>

6

7

>

.

.

.

¯

¯

>

6

7

.

.

.

<

6

7

U

¯

¯

u

u

u

n

=

i

i

i

2

3

4

3

6

7

¯

¯

6

7

>

>

6

7

.

.

.

¯

¯

>

.

.

.

>

6

7

.

.

.

¯

¯

>

>

6

7

>

¯

¯

>

4

5

>

>

¯

¯

:

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

Spectral Embeddings

  • Outline

  • Problem Overview

  • Spectral Embeddings

    • Basics

    • Problems

    • Solutions

  • Shape Descriptors

  • Results

  • Future Work

  • Acknowledgements

Examples of 3D embeddings


Spectral embeddings11 l.jpg

f

g

¾

¾

¾

k

1

2

;

;

:

:

:

;

1

³

´

2

k

k

2

u

i

¾

=

i

n

p

¸

i

=

r

P

¸

i

i

Spectral Embeddings

EigenValue Descriptor (EVD):

  • Use deviation in projected data as descriptor:

  • Our shape descriptor:

  • Outline

  • Problem Overview

  • Spectral Embeddings

    • Basics

    • Problems

    • Solutions

  • Shape Descriptors

  • Results

  • Future Work

  • Acknowledgements


Spectral embeddings12 l.jpg

2

3

b

b

b

2

3

¢

¢

¢

1

1

1

2

¢

¢

¢

1

a

a

a

1

1

1

2

1

n

n

2

R

A

b

¢

¢

¢

=

2

1

¢

¢

¢

a

6

7

2

1

6

7

6

7

T

6

7

R

A

A

.

.

.

6

7

=

.

.

.

6

7

.

.

.

.

.

.

6

7

.

.

.

B

6

7

.

.

.

A

=

6

7

=

b

b

b

6

7

¢

¢

¢

i

i

i

1

2

6

7

¢

¢

¢

a

a

a

n

i

i

i

1

2

6

7

n

6

7

R

E

E

¤

6

7

=

6

.

.

.

7

6

.

.

.

7

.

.

.

4

5

.

.

.

.

.

.

4

5

.

.

.

b

b

b

¢

¢

¢

1

2

n

n

n

n

¢

¢

¢

a

a

a

1

2

n

n

n

n

Spectral Embeddings

Why use eigenvalues??

  • Outline

  • Problem Overview

  • Spectral Embeddings

    • Basics

    • Problems

    • Solutions

  • Shape Descriptors

  • Results

  • Future Work

  • Acknowledgements


Spectral embeddings13 l.jpg

2

2

(

)

(

)

l

k

O

O

+

n

o

g

n

n

Spectral Embeddings

  • Outline

  • Problem Overview

  • Spectral Embeddings

    • Basics

    • Problems

    • Solutions

  • Shape Descriptors

  • Results

  • Future Work

  • Acknowledgements

Problems:

  • Geodesic distance computation

  • Efficiency of geodesic distance computation & eigendecomposition:


Spectral embeddings14 l.jpg

Spectral Embeddings

  • Outline

  • Problem Overview

  • Spectral Embeddings

    • Basics

    • Problems

    • Solutions

  • Shape Descriptors

  • Results

  • Future Work

  • Acknowledgements

Geodesics using Structural Graph:

  • Add edges to make mesh connected

  • Geodesic distance ≈ Shortest graph distance

    Problem: Unwanted (topology modifying) edges!

    Solution: Add shortest possible edges.

    Choice of graph to take edges from:

  • p-nearest neighbor (may not return connected graph)

  • p-edge connected [Yang 2004]


Spectral embeddings15 l.jpg

2

3

i

i

e

g

e

n

.

.

m

a

x

-

m

n

l

i

t

t

e

x

r

a

p

o

a

o

n

.

.

8

2

3

.

.

d

i

i

t

¯

¯

¯

¯

b

l

i

e

c

o

m

p

o

s

o

n

6

7

s

u

s

a

m

p

n

g

(

)

~

~

l

n

O

E

u

¢

¢

¢

u

n

>

=

¯

¯

¯

¯

l

l

l

1

£

6

7

3

(

)

l

O

>

(

)

l

l

O

z

}

|

{

>

4

5

n

o

g

n

¯

¯

¯

¯

.

.

.

>

6

7

8

2

3

.

.

>

.

.

.

¯

¯

¯

¯

¢

¢

¢

a

a

a

.

.

1

1

1

2

1

>

6

7

n

.

.

.

.

.

<

>

¯

¯

¯

¯

6

7

>

~

~

~

E

>

¢

¢

¢

a

u

u

u

n

2

1

=

¯

¯

¯

¯

2

3

4

3

£

>

6

7

6

7

n

>

>

6

7

¯

¯

¯

¯

6

7

>

>

.

.

.

>

6

7

¯

¯

¯

¯

>

6

.

.

.

7

<

.

.

.

>

.

.

.

6

7

.

.

.

A

>

4

5

¯

¯

¯

¯

n

.

.

.

=

>

£

6

7

n

n

>

¯

¯

¯

¯

¢

¢

¢

a

a

a

i

i

i

1

2

6

:

7

n

>

¯

¯

¯

¯

>

6

7

>

¯

¯

¯

¯

>

6

.

.

.

7

>

.

.

.

>

4

5

¯

¯

¯

¯

.

.

.

>

>

¯

¯

¯

¯

:

¢

¢

¢

a

a

a

1

2

n

n

n

n

¯

¯

¯

¯

¯

¯

¯

¯

n

z

}

|

{

8

2

3

¢

¢

¢

a

a

a

1

1

1

2

1

n

>

>

¢

¢

¢

a

a

a

2

1

2

2

2

<

n

6

7

l

A

=

l

l

£

6

7

.

.

.

4

5

.

.

.

>

.

.

.

>

:

¢

¢

¢

a

a

a

l

l

l

1

2

n

Spectral Embeddings

  • Outline

  • Problem Overview

  • Spectral Embeddings

    • Basics

    • Problems

    • Solutions

  • Shape Descriptors

  • Results

  • Future Work

  • Acknowledgements

Efficiency with Nyström approximation


Shape descriptor l.jpg

Shape Descriptor


Shape descriptor17 l.jpg

2

k

£

¤

p

¸

p

1

¡

°

i

i

X

(

)

S

C

P

Q

i

t

m

o

s

=

;

p

2

¸

p

+

°

i

i

i

1

=

Shape Descriptor

  • Outline

  • Problem Overview

  • Spectral Embeddings

  • Shape Descriptors

  • Results

  • Future Work

  • Acknowledgements

  • Global Shape Descriptors

    • Light Field Descriptor (LFD)

    • Spherical Harmonics Descriptor (SHD)

  • Our Similarity Measure (EVD):


Results l.jpg

Results


Experimental database l.jpg

Experimental Database

  • Outline

  • Problem Overview

  • Spectral Embeddings

  • Shape Descriptors

  • Results

  • Future Work

  • Acknowledgements

McGill 3D Articulated Shapes Database

http://www.cim.mcgill.ca/~shape/benchMark/


Results20 l.jpg

Precision-Recall plot for McGill database

Results

  • Outline

  • Problem Overview

  • Spectral Embeddings

  • Shape Descriptors

  • Results

  • Future Work

  • Acknowledgements


Results21 l.jpg

McGill articulated shape database

Results

  • Outline

  • Problem Overview

  • Spectral Embeddings

  • Shape Descriptors

  • Results

  • Future Work

  • Acknowledgements


Limitations future work l.jpg

Limitations & Future Work

  • Outline

  • Problem Overview

  • Spectral Embeddings

  • Shape Descriptors

  • Results

  • Future Work

  • Acknowledgements

  • Non-robustness of geodesic distances

  • Non-robustness to outliers


Acknowledgements l.jpg

Acknowledgements

  • Outline

  • Problem Overview

  • Spectral Embeddings

  • Shape Descriptors

  • Results

  • Future Work

  • Acknowledgements

  • McGill 3D Shape Benchmark.

  • Phil Shilane (LFD & SHD implementations).


Thank you for you attention l.jpg

Thank You! for you attention


  • Login