1 facial expression editing in video using a temporally smooth factorization
Download
Skip this Video
Download Presentation
1. Facial Expression Editing in Video Using a Temporally-Smooth Factorization

Loading in 2 Seconds...

play fullscreen
1 / 24

1. Facial Expression Editing in Video Using a Temporally-Smooth Factorization - PowerPoint PPT Presentation


  • 106 Views
  • Uploaded on

1. Facial Expression Editing in Video Using a Temporally-Smooth Factorization. 2. Face Swapping: Automatically Replacing Faces in Photographs. Facial Expression Editing in Video Using a Temporally-Smooth Factorization.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' 1. Facial Expression Editing in Video Using a Temporally-Smooth Factorization' - kyros


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
1 facial expression editing in video using a temporally smooth factorization

1. Facial Expression Editing in Video Using a Temporally-Smooth Factorization

2. Face Swapping: Automatically Replacing Faces in Photographs

facial expression editing in video using a temporally smooth factorization

Facial Expression Editing in Video Using a Temporally-Smooth Factorization

FeiYang, LubomirBourdev, Eli Shechtman, JueWang, DimitrisMetaxas

CVPR 2012

slide3
Goal

The goal is to allow for semantic-level editing of

expressions in a video:

  • magnifying an expression
  • suppressing an expression
  • replacing by another expressions
challenges
Challenges
  • Natural expression
    • Different parts changes accordingly
  • Unique identity
  • Temporal coherency
related work
Related Work
  • 2D based methods
    • [Theobald09], [Liu01], [Williams90], …
  • 3D based methods
    • [Blanz03], [Pighin98], …
  • Expression flow
    • [Yang11]…
  • Frame reorder method
    • [Bregler98], [Kemelmacher- Shlizerman11]
  • Tensor factorization methods
    • [Vlasic05], [Dale11]…
algorithm
Algorithm

3D Tensor Model

- [Vlasic et al siggraph05]

Modify

Expression Information

Identity Information

algorithm1
Algorithm

goal to identify a and

2D v.s. 3D

method

=

Weak Projective Matrix Rt

frame t

Minimize: | – |

algorithm2
Algorithm

Fitting Error:

Shape Distribution Constraint:

Temporal coherence:

algorithm3
Algorithm

Levenberg-Marquardt (Siggraph98)

algorithm4
Algorithm

Adjust to achieve expression modification

  • Dynamic Time Warping (DTW) [Sakoe78]
  • Residual Expression Flow
  • Correcting boundary compatibility
face swapping automatically replacing faces in photographs

Face Swapping: Automatically Replacing Faces in Photographs

Dmitri BitoukNeeraj Kumar SamreenDhillon Peter Belhumeur Shree K. Nayar

Siggraph 2008

goals
Goals

For an input image:

  • Automatically find the best candidate
  • Automatically replace the face
  • Automatically color and lighting adjustmet
library building
Library Building

OKAO face detector to detect face pose [Omron07]

alignment
Alignment

Pose, Resolution, and Image Blur:

  • Yaw, pitch threshold between two images ( )
  • Eye distance as a measure of distance (80%)
  • Similarity of the blur degrees [Kundur and Hatzinakos 1996; Fergus et al. 2006]
color and lighting
Color and Lighting

To ensure the similarity between the replaced and original face, a linear combination of 9 spherical harmonics [Ramamoorthi and Hanrahan 2001; Basri and Jacobs 2003] is used as measure metric:

Each pixel I(x, y) can be approximated by:

Distance:

seam signature
Seam Signature

256-by-256 patch from the face is used for replacement.

Unfold:

L2 Norm is used to compute the distance

appearance adjustment
Appearance Adjustment

Using simple scaling on the Harmonics coefficients

, are the original and replacement images

Scale the replaced image

the end
The End

Any Questions ?

ad