geppeto 1 a modeling approach to study the production of speech gestures
Download
Skip this Video
Download Presentation
GEPPETO 1 : A modeling approach to study the production of speech gestures

Loading in 2 Seconds...

play fullscreen
1 / 53

GEPPETO 1 : A modeling approach to study the production of speech gestures - PowerPoint PPT Presentation


  • 95 Views
  • Uploaded on

GEPPETO 1 : A modeling approach to study the production of speech gestures. Pascal Perrier (ICP – Grenoble) with Stéphanie Buchaillard (PhD) Matthieu Chabanas (ICP) Ma Liang (PhD), Yohan Payan (TIMC – Grenoble).

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' GEPPETO 1 : A modeling approach to study the production of speech gestures' - maris


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
geppeto 1 a modeling approach to study the production of speech gestures

GEPPETO1: A modeling approach to study the production of speech gestures

Pascal Perrier (ICP – Grenoble)

with

Stéphanie Buchaillard (PhD)

Matthieu Chabanas (ICP)

Ma Liang (PhD),

Yohan Payan (TIMC – Grenoble)

1GEstures shaped by thePhysicsand by aPErceptuallyorientedTargetsOptimization

outline
Outline
  • Introduction
  • Current hypotheses implemented in GEPPETO
  • Some results obtained with a 2D biomechanical tongue model
  • New issues raised by the use of 3D biomechanical tongue model
basic issues in speech production research
Basic issuesin Speech Production Research
  • Phonology/Phonetics Interface
    • Link between discrete representations and continuous physical signals
    • Nature of physical correlates of speech units
basic issues in speech production research1
Basic issuesin Speech Production Research
  • Control and Production of Speech Gestures
    • Control variables
    • Central representations of physical characteristics of the speech production apparatus
    • Interaction Perception-Action
basic issues in speech production research2
Basic issuesin Speech Production Research
  • From Gestures to Speech Sounds
    • Nature of acoustic sources
    • Relations between motor commands and acoustics
    • Interaction between airflow and articulatory gestures.
what is geppeto
What is GEPPETO?
  • An evolutive modeling framework to quantitatively test hypotheses about the control and the production of speech gestures.
  • It includes
    • Hypotheses about the physical correlates of phonological units.
    • Models of motor control
    • Physical models of the speech production apparatus
current hypotheses
Current Hypotheses
  • Phonology/Phonetic Interface
    • The smallest phonological unit is the phoneme
    • Phonemes are associated with target regions in the auditory domain
    • Larger phonological units are associated with speech sequences for which specific constraints exist for target optimization or for motor commands sequencing
current hypotheses1
Current Hypotheses
  • Control of speech gestures
    • Control variables: l commands (EP Hypothesis, Feldman, 1966)
    • No on line use of feedback going through the cortex.
    • Short-delay orosensory and proprioceptive feedbacks are taken into account.
    • Existence in the brain of internal representations of the speech apparatus (internal models).
current hypotheses2
Current Hypotheses
  • Control of speech gestures
    • Internal representations do not account for the whole physical complexity of the speech production apparatus
    • Kinematic characteristics are not directly controlled. They are the results of the interaction between motor control setups and physical phenomena of speech production
      • Which characteristics of speech signals are specifically controlled?
application to the generation of speech gestures with a 2 d biomechanical tongue model
Application to the generation of speech gestures with a 2 D biomechanical tongue model
  • Implementation of the model of control
  • Inversion from desired perceptual objectives to motor commands
  • Generation of gestures
2d biomechanical model
2D Biomechanical Model
  • Finite element structure
  • Linear elasticity (small deformations)
  • No account of the gravity
2d biomechanical model1

Posterior genioglossus

Anterior Genioglossus

Hyoglossus

2D Biomechanical Model
2d biomechanical model2

Styloglossus

Verticalis

Inferior Longitudinalis

2D Biomechanical Model
learning a static internal model from l commands to formants
Learning a static internal modelFrom l commands to formants
  • Step 1:
  • - Uniform sampling of
  • the l commands space
  • Generation of the
  • corresponding tongue
  • shapes.

9000 simulations

learning a static internal model from l commands to formants1
Learning a static internal modelFrom l commands to formants

Step 2: Computation of the area function.

learning a static internal model from l commands to formants2
Learning a static internal modelFrom l commands to formants

Step 3: Formants computation for 2 lip apertures

(red dots: spread lips; blue dots: rounded lips)

learning a static internal model from l commands to formants3

1st layer

Learning a static internal modelFrom l commands to formants

Step 4: Learning and generalizing

with radial basis functions

2nd layer

inversion from target regions to l commands

Target regions for some non rounded French phonemes

InversionFrom target regions to l commands
  • Target regions
  • Dispersion ellipses in the (F1, F2, F3) space
  • Currently defined by Fc1, Fc2, Fc3and sF1,sF2, sF3
inversion from target regions to l commands1

Target regions for some non rounded French phonemes

InversionFrom target regions to l commands
  • Target regions
  • Dispersion ellipses in the (F1, F2, F3) space
  • Currently defined by Fc1, Fc2, Fc3and sF1,sF2, sF3
inversion from target regions to l commands2

+

Speaker oriented

Listener oriented

InversionFrom target regions to l commands

Optimization

Cost minimization (Gradient descent technique)

Cost for a sequence made of N phonemes

with

inversion from target regions to l commands3
InversionFrom target regions to l commands

Example 1 Sequence [œ-e-k-i]

inversion from target regions to l commands4
InversionFrom target regions to l commands

Example 2 Sequence [œ-e-k-a]

production of tongue movements from inferred l commands

[oe] [e] [k] [a]

Production of tongue movements from inferred l commands

Serial command patterns

No difference between vowels and consonants

execution of tongue movements from inferred l commands
Execution of tongue movements from inferred l commands

Öhman’s model: Vowel-to-Vowel basis

Consonants are seen as perturbation of V-V

[oe] [e] [k] [a]

interaction control physics influence on the shapes of the articulatory paths
Interaction control / physics.Influence on the shapes of the articulatory paths

Example: the Articulatory loops

[aka]

[ika]

R. Houde (1969)

fluid wall interaction
Fluid-Wall Interaction

Imposed

pressure

difference

Forces

Mechanics of the

tissues.

Flow model

Finite

element model)

Deformation

interaction control physics influence on the shapes of the articulatory paths1

Deplacement X - Y

120

115

110

105

Y - mm

100

+++ PS = 3000 Pa

...... PS = 800 Pa

95

-------No aerodynamics

90

40

50

60

70

80

90

100

110

120

X - mm

Interaction control / physics.Influence on the shapes of the articulatory paths

Example: the Articulatory loops

[aka]

interaction control physics influence on the shapes of the articulatory paths2
Interaction control / physics.Influence on the shapes of the articulatory paths

Example: the Articulatory loops

No aerodynamics

With aerodynamics

[aka]

interaction control physics influence on the shapes of the articulatory paths3

Deplacement X - Y

113

112

... PS = 1600 Pa

---- No aerodynamics

111

110

Y - mm

109

108

107

61

62

63

64

65

66

67

X - mm

Interaction control / physics.Influence on the shapes of the articulatory paths

Example: the Articulatory loops

[ika]

interaction control physics influence on the shapes of the articulatory paths4
Interaction control / physics.Influence on the shapes of the articulatory paths

Example: the Articulatory loops

No aerodynamics

With aerodynamics

[ika]

a 3d biomechanical tongue model for a better account of physics
A 3D biomechanical tongue model:For a better account of physics
  • Visible Human Project ® data

(Wilhelms-Tricarico, 2003)

  • Finite Element Mesh made of Hexahedres
  • Adaptation of the mesh to a specific speaker (PB)

Gerard et al., ICP Grenoble

Wilhelms-Tricarico R.,1995

inner muscle structure of the tongue
Inner muscle structure of the tongue

Genioglossus (medium)

Genioglossus (anterior)

Styloglossus

Geniohyoid

Genioglossus (posterior)

Hyoglossus

Verticalis

Transversus

Inferior longitudinalis

Mylohyoid

Superior longitudinalis

vocal tract structure
Vocal tract structure

TONGUE’S BODY

HYOID BONE

MANDIBLE

PALATE

OTHER MUSCLES

elastical properties of tongue muscles

Displacement

0

Force

Linear

Non Linear

Tongue

Indentator

Elastical properties of tongue muscles
  • Hyperelastic material (2nd order Yeoh model) with large deformation hypothesis
dealing with gravity with the ep hypothesis1
Dealing with gravity with the EP hypothesis
  • Activation of GGp and MH
  •  Increase of reflex activity

[300ms]

dealing with gravity with the ep hypothesis3
Dealing with gravity with the EP hypothesis

Example of a good choice of control parameters

[300ms]

conclusions
Conclusions
  • A model of control based on perceptual objectives specified in terms of formants target regions associated with l motor commands and on an optimization process using a static model of the motor-perception relations can generate realistic speech movements if it is applying to a realistic physical model of speech production.
conclusions1
Conclusions
  • It supports our hypothesis that there is not need to assume the existence of a central optimization process that would apply to the articulatory trajectories in their whole (i.e. minimum of jerk, minimum of torque…)
conclusions2
Conclusions
  • It gives an interesting account of coarticulation phenomena by separating the effects of planning and those of physics.
  • It permits to test hypotheses about the phonological units (see serial model versus Öhman’s model).
conclusions3
Conclusions

However

  • a systematic comparison with data is required (currently in progress for French, German, Chinese, Japanese)
  • No account for time control, or for hypo/hyperspeech
  • No account for gravity
conclusions4
Conclusions
  • Necessity to work on a more complex internal representations that would integrate some aspects of articulatory dynamics.
influence of elasticity modeling
Influence of elasticity modeling

Hyperelastic

Large defo.

Linear

Small defo.

Linear

Activation of the Hyoglossus (2N)

ep hypothesis feldman 1966
EP Hypothesis(Feldman, 1966)

Perrier, Ostry, Laboissière, 1996

ep hypothesis feldman 19661
EP Hypothesis (Feldman, 1966)

Perrier, Ostry, Laboissière, 1996

static internal models

ld

Inverse Model

yi(t)

Direct Model

Static Internal Models

Central Nervous System

Desired

formants

Peripheral motor

system

l

Formants

ad