shape based retrieval of articulated 3d models using spectral embedding l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Shape-Based Retrieval of Articulated 3D Models Using Spectral Embedding PowerPoint Presentation
Download Presentation
Shape-Based Retrieval of Articulated 3D Models Using Spectral Embedding

Loading in 2 Seconds...

play fullscreen
1 / 24

Shape-Based Retrieval of Articulated 3D Models Using Spectral Embedding - PowerPoint PPT Presentation


  • 688 Views
  • Uploaded on

Shape-Based Retrieval of Articulated 3D Models Using Spectral Embedding Varun Jain and Hao Zhang {vjain,haoz}@cs.sfu.ca GrUVi Lab, School of Computing Science Simon Fraser University, Burnaby, BC Canada Problem Overview … … Problem Overview Database Outline Problem Overview

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Shape-Based Retrieval of Articulated 3D Models Using Spectral Embedding' - Faraday


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
shape based retrieval of articulated 3d models using spectral embedding

Shape-Based Retrieval of Articulated3D Models Using Spectral Embedding

Varun Jain and Hao Zhang

{vjain,haoz}@cs.sfu.ca

GrUVi Lab, School of Computing Science

Simon Fraser University, Burnaby, BC Canada

problem overview3

Problem Overview

Database

  • Outline
  • Problem Overview
    • Retrieval Problem
    • Methods
  • Spectral Embeddings
  • Shape Descriptors
  • Results
  • Future Work
  • Acknowledgements

Shape Retrieval

Applications

  • Computer aided design
  • Game design
  • Shape recognition
  • Face recognition

Query

Interface

User

Output

shape retrieval
Shape Retrieval
  • Outline
  • Problem Overview
    • Retrieval Problem
    • Methods
  • Spectral Embeddings
  • Shape Descriptors
  • Results
  • Future Work
  • Acknowledgements

How?

  • Using Correspondence

Efficiency??

  • Using Global Descriptors

For 3D shapes

    • Fourier Descriptors
    • Light Field
    • Spherical Harmonics
    • Skeletal Graph Matching
  • Non-rigid transformations??
  • Stretching
  • Articulation (bending)
shape retrieval5
Shape Retrieval
  • Outline
  • Problem Overview
    • Retrieval Problem
    • Methods
  • Spectral Embeddings
  • Shape Descriptors
  • Results
  • Future Work
  • Acknowledgements

Our Method

  • Normalize non-rigid transformations
    • Construct affinity matrix
    • Spectral embedding
  • Use global shape descriptors
    • Light Field Descriptor (LFD)
    • Spherical Harmonics Descriptor (SHD)
    • Eigenvalues??
shape retrieval6
Shape Retrieval
  • Outline
  • Problem Overview
    • Retrieval Problem
    • Methods
  • Spectral Embeddings
  • Shape Descriptors
  • Results
  • Future Work
  • Acknowledgements

Advantages of Our Method

  • Handles shape articulation (best performance for articulated shapes).
  • Flexibility of affinity matrices
  • Robustness of affinity matrices
spectral embeddings8

2

(

)

d

i

j

n

g

e

o

;

2

a

e

2

=

¾

i

j

a

a

a

1

1

1

2

1

¢

¢

¢

n

z

}

|

{

a

8

2

3

2

1

¢

¢

¢

>

.

.

.

>

.

.

.

>

>

6

7

.

.

.

A

=

>

n

>

6

7

>

a

a

a

>

6

7

n

n

<

i

i

i

1

2

6

7

¢

¢

¢

n

£

.

.

.

6

7

.

.

.

6

7

.

.

.

>

>

6

7

>

a

a

a

>

6

7

>

>

4

5

1

2

¢

¢

¢

n

n

n

n

>

>

:

Spectral Embeddings

Affinity matrix

  • Outline
  • Problem Overview
  • Spectral Embeddings
    • Basics
    • Problems
    • Solutions
  • Shape Descriptors
  • Results
  • Future Work
  • Acknowledgements
spectral embeddings9

¯

¯

¯

T

A

E

¤

E

¯

¯

¯

=

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

k

¯

¯

¯

¯

¯

¯

k-dimensional spectral embedding coordinates of ith point of P

z

}

|

{

~

~

~

¯

¯

¯

u

u

¢

¢

¢

u

k

1

2

¯

¯

¯

8

2

3

¯

¯

¯

¯

¯

¯

>

>

>

>

6

7

>

>

6

7

>

.

.

.

>

6

7

>

.

.

.

>

6

7

.

.

.

1

<

6

7

U

E

¤

2

¢

¢

¢

u

u

u

n

=

=

k

k

k

i

i

i

1

2

6

7

k

6

7

>

>

6

.

.

.

7

>

.

.

.

>

6

7

.

.

.

>

>

6

7

>

>

4

5

>

>

:

Spectral Embeddings

Eigenvalue decomposition:

Scaled eigenvectors:

  • Outline
  • Problem Overview
  • Spectral Embeddings
    • Basics
    • Problems
    • Solutions
  • Shape Descriptors
  • Results
  • Future Work
  • Acknowledgements

Jain, V., Zhang, H.: Robust 3D Shape Correspondence in the Spectral Domain. Proc. Shape Modeling International 2006.

spectral embeddings10

~

~

~

u

u

u

2

3

4

8

2

3

¯

¯

>

¯

¯

>

>

>

6

7

¯

¯

>

>

6

7

>

¯

¯

.

.

.

>

6

7

>

.

.

.

¯

¯

>

6

7

.

.

.

<

6

7

U

¯

¯

u

u

u

n

=

i

i

i

2

3

4

3

6

7

¯

¯

6

7

>

>

6

7

.

.

.

¯

¯

>

.

.

.

>

6

7

.

.

.

¯

¯

>

>

6

7

>

¯

¯

>

4

5

>

>

¯

¯

:

¯

¯

¯

¯

¯

¯

¯

¯

¯

¯

Spectral Embeddings
  • Outline
  • Problem Overview
  • Spectral Embeddings
    • Basics
    • Problems
    • Solutions
  • Shape Descriptors
  • Results
  • Future Work
  • Acknowledgements

Examples of 3D embeddings

spectral embeddings11

f

g

¾

¾

¾

k

1

2

;

;

:

:

:

;

1

³

´

2

k

k

2

u

i

¾

=

i

n

p

¸

i

=

r

P

¸

i

i

Spectral Embeddings

EigenValue Descriptor (EVD):

  • Use deviation in projected data as descriptor:
  • Our shape descriptor:
  • Outline
  • Problem Overview
  • Spectral Embeddings
    • Basics
    • Problems
    • Solutions
  • Shape Descriptors
  • Results
  • Future Work
  • Acknowledgements
spectral embeddings12

2

3

b

b

b

2

3

¢

¢

¢

1

1

1

2

¢

¢

¢

1

a

a

a

1

1

1

2

1

n

n

2

R

A

b

¢

¢

¢

=

2

1

¢

¢

¢

a

6

7

2

1

6

7

6

7

T

6

7

R

A

A

.

.

.

6

7

=

.

.

.

6

7

.

.

.

.

.

.

6

7

.

.

.

B

6

7

.

.

.

A

=

6

7

=

b

b

b

6

7

¢

¢

¢

i

i

i

1

2

6

7

¢

¢

¢

a

a

a

n

i

i

i

1

2

6

7

n

6

7

R

E

E

¤

6

7

=

6

.

.

.

7

6

.

.

.

7

.

.

.

4

5

.

.

.

.

.

.

4

5

.

.

.

b

b

b

¢

¢

¢

1

2

n

n

n

n

¢

¢

¢

a

a

a

1

2

n

n

n

n

Spectral Embeddings

Why use eigenvalues??

  • Outline
  • Problem Overview
  • Spectral Embeddings
    • Basics
    • Problems
    • Solutions
  • Shape Descriptors
  • Results
  • Future Work
  • Acknowledgements
spectral embeddings13

2

2

(

)

(

)

l

k

O

O

+

n

o

g

n

n

Spectral Embeddings
  • Outline
  • Problem Overview
  • Spectral Embeddings
    • Basics
    • Problems
    • Solutions
  • Shape Descriptors
  • Results
  • Future Work
  • Acknowledgements

Problems:

  • Geodesic distance computation
  • Efficiency of geodesic distance computation & eigendecomposition:
spectral embeddings14
Spectral Embeddings
  • Outline
  • Problem Overview
  • Spectral Embeddings
    • Basics
    • Problems
    • Solutions
  • Shape Descriptors
  • Results
  • Future Work
  • Acknowledgements

Geodesics using Structural Graph:

  • Add edges to make mesh connected
  • Geodesic distance ≈ Shortest graph distance

Problem: Unwanted (topology modifying) edges!

Solution: Add shortest possible edges.

Choice of graph to take edges from:

  • p-nearest neighbor (may not return connected graph)
  • p-edge connected [Yang 2004]
spectral embeddings15

2

3

i

i

e

g

e

n

.

.

m

a

x

-

m

n

l

i

t

t

e

x

r

a

p

o

a

o

n

.

.

8

2

3

.

.

d

i

i

t

¯

¯

¯

¯

b

l

i

e

c

o

m

p

o

s

o

n

6

7

s

u

s

a

m

p

n

g

(

)

~

~

l

n

O

E

u

¢

¢

¢

u

n

>

=

¯

¯

¯

¯

l

l

l

1

£

6

7

3

(

)

l

O

>

(

)

l

l

O

z

}

|

{

>

4

5

n

o

g

n

¯

¯

¯

¯

.

.

.

>

6

7

8

2

3

.

.

>

.

.

.

¯

¯

¯

¯

¢

¢

¢

a

a

a

.

.

1

1

1

2

1

>

6

7

n

.

.

.

.

.

<

>

¯

¯

¯

¯

6

7

>

~

~

~

E

>

¢

¢

¢

a

u

u

u

n

2

1

=

¯

¯

¯

¯

2

3

4

3

£

>

6

7

6

7

n

>

>

6

7

¯

¯

¯

¯

6

7

>

>

.

.

.

>

6

7

¯

¯

¯

¯

>

6

.

.

.

7

<

.

.

.

>

.

.

.

6

7

.

.

.

A

>

4

5

¯

¯

¯

¯

n

.

.

.

=

>

£

6

7

n

n

>

¯

¯

¯

¯

¢

¢

¢

a

a

a

i

i

i

1

2

6

:

7

n

>

¯

¯

¯

¯

>

6

7

>

¯

¯

¯

¯

>

6

.

.

.

7

>

.

.

.

>

4

5

¯

¯

¯

¯

.

.

.

>

>

¯

¯

¯

¯

:

¢

¢

¢

a

a

a

1

2

n

n

n

n

¯

¯

¯

¯

¯

¯

¯

¯

n

z

}

|

{

8

2

3

¢

¢

¢

a

a

a

1

1

1

2

1

n

>

>

¢

¢

¢

a

a

a

2

1

2

2

2

<

n

6

7

l

A

=

l

l

£

6

7

.

.

.

4

5

.

.

.

>

.

.

.

>

:

¢

¢

¢

a

a

a

l

l

l

1

2

n

Spectral Embeddings
  • Outline
  • Problem Overview
  • Spectral Embeddings
    • Basics
    • Problems
    • Solutions
  • Shape Descriptors
  • Results
  • Future Work
  • Acknowledgements

Efficiency with Nyström approximation

shape descriptor17

2

k

£

¤

p

¸

p

1

¡

°

i

i

X

(

)

S

C

P

Q

i

t

m

o

s

=

;

p

2

¸

p

+

°

i

i

i

1

=

Shape Descriptor
  • Outline
  • Problem Overview
  • Spectral Embeddings
  • Shape Descriptors
  • Results
  • Future Work
  • Acknowledgements
  • Global Shape Descriptors
    • Light Field Descriptor (LFD)
    • Spherical Harmonics Descriptor (SHD)
  • Our Similarity Measure (EVD):
experimental database
Experimental Database
  • Outline
  • Problem Overview
  • Spectral Embeddings
  • Shape Descriptors
  • Results
  • Future Work
  • Acknowledgements

McGill 3D Articulated Shapes Database

http://www.cim.mcgill.ca/~shape/benchMark/

results20

Precision-Recall plot for McGill database

Results
  • Outline
  • Problem Overview
  • Spectral Embeddings
  • Shape Descriptors
  • Results
  • Future Work
  • Acknowledgements
results21

McGill articulated shape database

Results
  • Outline
  • Problem Overview
  • Spectral Embeddings
  • Shape Descriptors
  • Results
  • Future Work
  • Acknowledgements
limitations future work
Limitations & Future Work
  • Outline
  • Problem Overview
  • Spectral Embeddings
  • Shape Descriptors
  • Results
  • Future Work
  • Acknowledgements
  • Non-robustness of geodesic distances
  • Non-robustness to outliers
acknowledgements
Acknowledgements
  • Outline
  • Problem Overview
  • Spectral Embeddings
  • Shape Descriptors
  • Results
  • Future Work
  • Acknowledgements
  • McGill 3D Shape Benchmark.
  • Phil Shilane (LFD & SHD implementations).