Loading in 5 sec....

Face Transfer with Multilinear Models PowerPoint Presentation

Face Transfer with Multilinear Models

- 119 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about ' Face Transfer with Multilinear Models ' - zamora

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

### Face Transfer with Multilinear Models

Daniel Vlasic & Jovan Popovic

CSAIL MIT

Matthew Brand & Hanspeter Pfister

MERL

Outline

- Introduction to Multilinear Model
- Multilinear Face Model
- Face Transfer

I2

X1

X2

X1-X2

X1

X2

I1

B

Y1

Y2

Y1-Y2

Y1

Y2

X1-Y1

X2-Y2

(X1-Y1) –

(X2-Y2)

X1

X2

1

0

X1

X2

Y1

Y2

=

1

0

x

Y1

Y2

1

-1

X1-Y1

X2-Y2

U(1)

A = (U(2)x(U(1)xB)T)T

1

0

X1

Y1

X1-Y1

AT

1

0

=

x

A =B x1U(1) x2U(2)

X2

Y2

X2-Y2

1

-1

U(2)

Multilinear Model

- Generalization of linear model
- A = B x1U(1)x2U(2)x3U(3)…xnU(n)…xNU(N)

Orthogonal

Transformation

Data Tensor

Core Tensor

How to Multiply?

- A = B x1 U(1)x2 U(2)x3 U(3)…xn U(n)…
- B xn U(n) :U(n) * B(n)

I2

Tensor Flattening

X1

X2

I1

B

A =B x1U(1) x2U(2)

Y1

Y2

Face in Multilinear Model

Data Tensor

Mathematically… PCA => Find Cov(A) = (AAT)/(n-1)

?

Data

Tensor

Left Singular of SVD

In data reduction, we use PCA as Y = eTX

- SVD => A = USVT
- AAT = USVT(USVT)T = USVT * ((VT)TSUT) = US2UT

- AAT = eDeT => U = e

SVD for Multilinear Model

- To find Un, perform SVD on mode n space of the data tensor, i.e., J(n)
- This is not optimal, however, and they use ALS, or Alternating Least Square
- A lot of SIAM papers address this topic, and out of our scope

Multilinear Face Model Trilinear Model (4-mode)

- Bilinear Model (3-mode)
- 30K vertices x 10 expression x 15 identities

- + 5 visemes

Multilinear model of face geometry

Arbitrary Interpolation

n

Synthesized Data, f

1

=

n

Original Data, M

m

Weighting, w

m rows data

1

x

f= Mx2w(2)

Interpolation in Multilinear Model

F = M x2w(2)

Multilinear model of face geometry

f = M x2 w(2) x3 w(3) x4 w(4) …. xN w(N)

Missing Data

- So far, we dealt with perfect data set
- In practice… NOT the case
- Maximum A Posteriori (MAP) estimation failed
- Probability Principle Component Analysis (PPCA)

Short Review on PPCA

- t = Wx +μ +ε
- x is N(0, I) , εis isotropic error N(0, σ2I)
- So t is N(μ, WWT + σ2I)
- Given t, we want to estimate W, σ
- Maximize the likelihood function L = p(t) = Πip(ti |x,W)

Short Review on PPCA

- Maximum Likelihood Estimators (M.L.E) tells us that, by taking log-likelihood
- WML = Uq(Λq – σ2I)1/2R
- σML = 1/(d-q) Σj = q+1 to dλj
- Uq is eigen-vector and Λq eigen-value
- -------------------------------------------------------
- End of review

Face Tracking

- Kanade-Lucas-Tomasi (KLT) algorithm
- Zd = Z(p – p0) = e
- Z(sR fi + t - po) = e for vertex i
- Z(sR Mm,iwm + t – p0) = e

Download Presentation

Connecting to Server..