1 facial expression editing in video using a temporally smooth factorization
Download
1 / 24

1. Facial Expression Editing in Video Using a Temporally-Smooth Factorization - PowerPoint PPT Presentation


  • 106 Views
  • Uploaded on

1. Facial Expression Editing in Video Using a Temporally-Smooth Factorization. 2. Face Swapping: Automatically Replacing Faces in Photographs. Facial Expression Editing in Video Using a Temporally-Smooth Factorization.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about '1. Facial Expression Editing in Video Using a Temporally-Smooth Factorization' - kyros


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
1 facial expression editing in video using a temporally smooth factorization

1. Facial Expression Editing in Video Using a Temporally-Smooth Factorization

2. Face Swapping: Automatically Replacing Faces in Photographs


Facial expression editing in video using a temporally smooth factorization

Facial Expression Editing in Video Using a Temporally-Smooth Factorization

FeiYang, LubomirBourdev, Eli Shechtman, JueWang, DimitrisMetaxas

CVPR 2012


1 facial expression editing in video using a temporally smooth factorization
Goal Factorization

The goal is to allow for semantic-level editing of

expressions in a video:

  • magnifying an expression

  • suppressing an expression

  • replacing by another expressions


Example
Example Factorization


Challenges
Challenges Factorization

  • Natural expression

    • Different parts changes accordingly

  • Unique identity

  • Temporal coherency


Related work
Related Work Factorization

  • 2D based methods

    • [Theobald09], [Liu01], [Williams90], …

  • 3D based methods

    • [Blanz03], [Pighin98], …

  • Expression flow

    • [Yang11]…

  • Frame reorder method

    • [Bregler98], [Kemelmacher- Shlizerman11]

  • Tensor factorization methods

    • [Vlasic05], [Dale11]…


Algorithm
Algorithm Factorization

3D Tensor Model

- [Vlasic et al siggraph05]

Modify

Expression Information

Identity Information


Mode n product
Mode-n Product Factorization


Algorithm1
Algorithm Factorization

goal to identify a and

2D v.s. 3D

method

=

Weak Projective Matrix Rt

frame t

Minimize: | – |


Algorithm2
Algorithm Factorization

Fitting Error:

Shape Distribution Constraint:

Temporal coherence:


Algorithm3
Algorithm Factorization

Levenberg-Marquardt (Siggraph98)


Algorithm4
Algorithm Factorization

Adjust to achieve expression modification

  • Dynamic Time Warping (DTW) [Sakoe78]

  • Residual Expression Flow

  • Correcting boundary compatibility


Results
Results Factorization


Face swapping automatically replacing faces in photographs

Face Swapping: Automatically Replacing Faces in Photographs Factorization

Dmitri BitoukNeeraj Kumar SamreenDhillon Peter Belhumeur Shree K. Nayar

Siggraph 2008


Examples
Examples Factorization


Goals
Goals Factorization

For an input image:

  • Automatically find the best candidate

  • Automatically replace the face

  • Automatically color and lighting adjustmet


Library building
Library Building Factorization

OKAO face detector to detect face pose [Omron07]


Process
Process Factorization


Alignment
Alignment Factorization

Pose, Resolution, and Image Blur:

  • Yaw, pitch threshold between two images ( )

  • Eye distance as a measure of distance (80%)

  • Similarity of the blur degrees [Kundur and Hatzinakos 1996; Fergus et al. 2006]


Color and lighting
Color and Lighting Factorization

To ensure the similarity between the replaced and original face, a linear combination of 9 spherical harmonics [Ramamoorthi and Hanrahan 2001; Basri and Jacobs 2003] is used as measure metric:

Each pixel I(x, y) can be approximated by:

Distance:


Seam signature
Seam Signature Factorization

256-by-256 patch from the face is used for replacement.

Unfold:

L2 Norm is used to compute the distance


Appearance adjustment
Appearance Adjustment Factorization

Using simple scaling on the Harmonics coefficients

, are the original and replacement images

Scale the replaced image


Results1
Results Factorization


The end
The End Factorization

Any Questions ?