human emotion synthesis n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Human Emotion Synthesis PowerPoint Presentation
Download Presentation
Human Emotion Synthesis

Loading in 2 Seconds...

play fullscreen
1 / 34

Human Emotion Synthesis - PowerPoint PPT Presentation


  • 107 Views
  • Uploaded on

Human Emotion Synthesis. David Oziem, Lisa Gralewski , Neill Campbell, Colin Dalton, David Gibson, Barry Thomas University of Bristol, Motion Ripper, 3CR Research. Project Group. Motion Ripper Project Methods of motion capture. Re-using captured motion signatures.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Human Emotion Synthesis' - zelig


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
human emotion synthesis

Human Emotion Synthesis

David Oziem, Lisa Gralewski, Neill Campbell, Colin Dalton, David Gibson, Barry Thomas

University of Bristol, Motion Ripper, 3CR Research

project group
Project Group
  • Motion Ripper Project
    • Methods of motion capture.
    • Re-using captured motion signatures.
    • Synthesising new or extend motion sequences.
    • Tools to aid animation.
  • Collaboration between University of Bristol CS, Matrix Media & Granada.

Synthesising Facial Emotions – University of Bristol – 3CR Research

introduction
Introduction
  • What is an emotion?
  • Ekman outlined 6 different basic emotions.
    • joy, disgust, surprise, fear, anger and sadness.
  • Emotional states relate to ones expression and movement.
  • Synthesising video footage of an actress expressing different emotions.

Synthesising Facial Emotions – University of Bristol – 3CR Research

video textures
Video Textures
  • Video textures or temporal textures are textures with motion. (Szummer’96)
  • Schodl’00, reordered frames from the original to produce loops or continuous sequences.
    • Doesn’t produce new footage.
  • Campbell’01, Fitzgibbon’01, Reissell’01, used Autoregressive process (ARP) to synthesis frames.

Examples of Video Textures

Synthesising Facial Emotions – University of Bristol – 3CR Research

autoregressive process
Autoregressive Process
  • Statistical model
  • Calculating the model involves working out the parameter vector (a1…an) and w.
  • n is known as the order of the sequence.

Current value at time t

y(t) = – a1y(t – 1) – a2y(t – 2) – … – any(t – n) + w.ε

Parameter vector (a1,…,an)

Noise

Synthesising Facial Emotions – University of Bristol – 3CR Research

autoregressive process1
Autoregressive Process
  • Statistical model
  • Increasing dimensionality of y drastically increases the complexity in calculating (a1…an).

y(t) = – a1y(t – 1) – a2y(t – 2) – … – any(t – n) + w.ε

Synthesising Facial Emotions – University of Bristol – 3CR Research

autoregressive process2
Autoregressive Process

Secondary mode

Primary mode

PCA analysis of Sad footage in 2D

  • Principal Components Analysis is used to reduce number of dimensions in the original sequence.

Synthesising Facial Emotions – University of Bristol – 3CR Research

autoregressive process3
Autoregressive Process

Secondary mode

Secondary mode

Primary mode

Primary mode

PCA analysis of Sad footage in 2D

Generated sequence using an ARP

  • Non-Gaussian Distributionis incorrectly modelled by an ARP.

Synthesising Facial Emotions – University of Bristol – 3CR Research

face modelling
Face Modelling
  • Campbell’01, synthesised a talking head.
  • Cootes and Talyor’00, combined appearance model.
    • Isolates shape and texture.
  • Requires labelled frames.
    • Must label important features on the face.

Labelled points

Synthesising Facial Emotions – University of Bristol – 3CR Research

combined appearance
Combined Appearance

Shape space

Hand Labelled video footage provides a point set which represents the shape space of the clip.

Synthesising Facial Emotions – University of Bristol – 3CR Research

combined appearance1
Combined Appearance

Shape space

Texture space

Warping each frame into a standard pose, creates the texture space.

The standard pose is the mean position of the points.

Synthesising Facial Emotions – University of Bristol – 3CR Research

combined appearance2
Combined Appearance

Shape space

Texture space

Combined space

Joining the shape and texture space and then re-analysing using PCA produces the combined space.

Synthesising Facial Emotions – University of Bristol – 3CR Research

combined appearance3
Combined Appearance

Shape space

Texture space

Combined space

Reconstruction of the original sequence from the combined space.

Combined space

Synthesising Facial Emotions – University of Bristol – 3CR Research

combined appearance4
Combined Appearance

Secondary mode

Secondary mode

Primary mode

Original sequence in 2D

Change in distribution after applying

The combined appearance technique

Primary mode

Combined Appearance sequence

Synthesising Facial Emotions – University of Bristol – 3CR Research

combined appearance5
Combined Appearance
  • Visually the generated plot appears to have been generated using the same stochastic process as the original.

Secondary mode

Secondary mode

ARP

model

Primary mode

Primary mode

Generated Sequence

Original sequence

Synthesising Facial Emotions – University of Bristol – 3CR Research

copying and arp
Copying and ARP
  • Combine the benefits of copying with ARP
    • New motion signatures.
    • Handles non-Gaussian distributions.

Synthesising Facial Emotions – University of Bristol – 3CR Research

copying and arp1
Important to reduce the complexity of the search process.

Need around 30 to 40 dimensions in this example.

Copying and ARP

Original input

PCA

Reduced input

Synthesising Facial Emotions – University of Bristol – 3CR Research

copying and arp2
Temporal segments of between 15 to 30 frames.

Need to reduce each segment to be able to train ARP’s.

Copying and ARP

Original input

PCA

Segmented input

Reduced segments

PCA

Reduced input

Synthesising Facial Emotions – University of Bristol – 3CR Research

copying and arp3
Many of the learned models are unstable.

10-20% are usable.

Copying and ARP

Original input

PCA

Segmented input

Reduced segments

PCA

ARP

Reduced input

Synthesised segments

Synthesising Facial Emotions – University of Bristol – 3CR Research

copying and arp4
Copying and ARP

Original input

PCA

Segmented input

Reduced segments

PCA

ARP

Reduced input

Segment selection

Synthesised segments

Outputted Sequence

Synthesising Facial Emotions – University of Bristol – 3CR Research

example
Example

First mode

Possible segments.

End of generated sequence.

Compared section

Time t

Synthesising Facial Emotions – University of Bristol – 3CR Research

example1
Example

First mode

Closest 3 segments

are chosen.

Time t

Synthesising Facial Emotions – University of Bristol – 3CR Research

example2
Example

First mode

The segment to be copied is randomly selected from the closest 3.

Time t

Synthesising Facial Emotions – University of Bristol – 3CR Research

example3
Example

First mode

Segments are blended together using a small overlap and averaging the overlapping pixels.

Time t

Synthesising Facial Emotions – University of Bristol – 3CR Research

copying and arp5
Copying and ARP
  • Potentially infinitely long.
  • Includes new novel motions.

Secondary mode

Secondary mode

Copying

& ARP

model

Primary mode

Primary mode

PCA analysis of Sad

footage in 2D

Generated sequence

Synthesising Facial Emotions – University of Bristol – 3CR Research

results angry
Results (Angry)
  • Combined appearance produces higher resolution frames.
  • Better motion from the copying and ARP approach

Source Footage

Combined Appearance ARP

Copying with ARP

Synthesising Facial Emotions – University of Bristol – 3CR Research

results sad
Results (Sad)
  • Similar results as with the angry footage
    • Copied approach is less blurred due to the reduced variance.

Source Footage

Combined Appearance ARP

Copying with ARP

Synthesising Facial Emotions – University of Bristol – 3CR Research

comparison results
Comparison Results
  • Simple objective comparison.
    • Randomly selected temporal segments.

- Combined appearance - Segment copying

Synthesising Facial Emotions – University of Bristol – 3CR Research

comparison
Comparison
  • Perceptually is it better to have good motion or higher resolution.

Synthesising Facial Emotions – University of Bristol – 3CR Research

slide31

Combined appearance

Segment Copying with ARP

Synthesising Facial Emotions – University of Bristol – 3CR Research

other potential uses
Other potential uses
  • Self Organising Map
  • Uses combined appearance
    • as each ARP model provides a minimal representation of the given emotion.
  • Can navigate between emotions to create new interstates.

Angry SadHappy

Synthesising Facial Emotions – University of Bristol – 3CR Research

conclusions
Conclusions
  • Both methods can produce synthesised clips of a given emotion.
  • Combined appearance produces higher definition frames.
  • Copying and ARPs generates more natural movements.

Synthesising Facial Emotions – University of Bristol – 3CR Research

questions
Questions

Synthesising Facial Emotions – University of Bristol – 3CR Research