1 facial expression editing in video using a temporally smooth factorization
This presentation is the property of its rightful owner.
Sponsored Links
1 / 24

1. Facial Expression Editing in Video Using a Temporally-Smooth Factorization PowerPoint PPT Presentation


  • 76 Views
  • Uploaded on
  • Presentation posted in: General

1. Facial Expression Editing in Video Using a Temporally-Smooth Factorization. 2. Face Swapping: Automatically Replacing Faces in Photographs. Facial Expression Editing in Video Using a Temporally-Smooth Factorization.

Download Presentation

1. Facial Expression Editing in Video Using a Temporally-Smooth Factorization

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


1 facial expression editing in video using a temporally smooth factorization

1. Facial Expression Editing in Video Using a Temporally-Smooth Factorization

2. Face Swapping: Automatically Replacing Faces in Photographs


Facial expression editing in video using a temporally smooth factorization

Facial Expression Editing in Video Using a Temporally-Smooth Factorization

FeiYang, LubomirBourdev, Eli Shechtman, JueWang, DimitrisMetaxas

CVPR 2012


1 facial expression editing in video using a temporally smooth factorization

Goal

The goal is to allow for semantic-level editing of

expressions in a video:

  • magnifying an expression

  • suppressing an expression

  • replacing by another expressions


Example

Example


Challenges

Challenges

  • Natural expression

    • Different parts changes accordingly

  • Unique identity

  • Temporal coherency


Related work

Related Work

  • 2D based methods

    • [Theobald09], [Liu01], [Williams90], …

  • 3D based methods

    • [Blanz03], [Pighin98], …

  • Expression flow

    • [Yang11]…

  • Frame reorder method

    • [Bregler98], [Kemelmacher- Shlizerman11]

  • Tensor factorization methods

    • [Vlasic05], [Dale11]…


Algorithm

Algorithm

3D Tensor Model

- [Vlasic et al siggraph05]

Modify

Expression Information

Identity Information


Mode n product

Mode-n Product


Algorithm1

Algorithm

goal to identify a and

2D v.s. 3D

method

=

Weak Projective Matrix Rt

frame t

Minimize: | – |


Algorithm2

Algorithm

Fitting Error:

Shape Distribution Constraint:

Temporal coherence:


Algorithm3

Algorithm

Levenberg-Marquardt (Siggraph98)


Algorithm4

Algorithm

Adjust to achieve expression modification

  • Dynamic Time Warping (DTW) [Sakoe78]

  • Residual Expression Flow

  • Correcting boundary compatibility


Results

Results


Face swapping automatically replacing faces in photographs

Face Swapping: Automatically Replacing Faces in Photographs

Dmitri BitoukNeeraj Kumar SamreenDhillon Peter Belhumeur Shree K. Nayar

Siggraph 2008


Examples

Examples


Goals

Goals

For an input image:

  • Automatically find the best candidate

  • Automatically replace the face

  • Automatically color and lighting adjustmet


Library building

Library Building

OKAO face detector to detect face pose [Omron07]


Process

Process


Alignment

Alignment

Pose, Resolution, and Image Blur:

  • Yaw, pitch threshold between two images ( )

  • Eye distance as a measure of distance (80%)

  • Similarity of the blur degrees [Kundur and Hatzinakos 1996; Fergus et al. 2006]


Color and lighting

Color and Lighting

To ensure the similarity between the replaced and original face, a linear combination of 9 spherical harmonics [Ramamoorthi and Hanrahan 2001; Basri and Jacobs 2003] is used as measure metric:

Each pixel I(x, y) can be approximated by:

Distance:


Seam signature

Seam Signature

256-by-256 patch from the face is used for replacement.

Unfold:

L2 Norm is used to compute the distance


Appearance adjustment

Appearance Adjustment

Using simple scaling on the Harmonics coefficients

, are the original and replacement images

Scale the replaced image


Results1

Results


The end

The End

Any Questions ?


  • Login