Download
markup of multimodal emotion sensitive corpora n.
Skip this Video
Loading SlideShow in 5 Seconds..
Markup of Multimodal Emotion-Sensitive Corpora PowerPoint Presentation
Download Presentation
Markup of Multimodal Emotion-Sensitive Corpora

Markup of Multimodal Emotion-Sensitive Corpora

106 Views Download Presentation
Download Presentation

Markup of Multimodal Emotion-Sensitive Corpora

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Markup of Multimodal Emotion-Sensitive Corpora Berardina Nadja de Carolis, Univ. Bari Marc Schröder, DFKI

  2. Our Background:Generation of ECA Behavior • Believable ECA: multimodality, affect, personality, human-like, etc. • Two approaches: • Bari APML (Affective Presentation Markup Language) • NECA RRL (Rich Representation Language)

  3. Overview • Markup Schema Definition • What to Annotate • What is a Markup Language • What is XML • Dealing with complexity of human communication • Labelling Tools

  4. Markup Scheme Definition • What to annotate? (verbal, non-verbal behavior, emotions, context, …) • Units of consideration (utterances/ turns/ paragraphs/ time units/…) • Descriptive Scheme • Categorical • Dimensional • Functional • …

  5. The Question of Coding • How to Share and Access Data Annotation? • Standardization • Format Readable for Computers (statistics, inference, …) • XML-based Markup Languages!

  6. What is a Markup-Language • Dictionary Definition: a set of symbols and rules for their use when doing a markup of a document nomenclature, terminology, language - a system of words used in a particular discipline; • Computer Domain Examples: • HTML • LATEX • RTF • … • But … • proprietary language • not generic • not standard • fixed syntax and semantics <?XML!>

  7. <pitch level=“high”> <emotion type=“reprimanding”> </pitch> </emotion> What is XML XML (eXtensible Markup Language) is • a language for creating markup languages that describe data thus, a meta language used to define new markup languages (domain- or industry-specific). • Simple, open standard, extensible, ready to use, … XML has a lot of friends that makes computation easy (XSL, DOM, SAX, …) and interoperable (RDF, RDF Schema, OWL, …) No No No! I said turn left!!!

  8. Dealing with complexity of human communication • Communication at different non-exclusive abstraction levels • Mental state (Beliefs, Goals, Intentions, Affective State, …) • Discourse (Communicative Goals, Focus, Relations) • Meanings (Performative, Affect, Turn-Taking, Emphasis) • Signals (Physical Communicative Expressions) • Defining Different Markup Layers • Separate levels of meaning into different layers • Define a Markup Language for each layer • Define Interaction between layers • Develop Easy-to-use tools

  9. Labelling tools:Text-centered annotation: MMAX http://mmax.eml-research.de

  10. Labelling tools:Video-centered annotation: ANVIL http://www.dfki.de/~kipp/anvil

  11. Feedback into Generation Systems • Annotated Corpora can be used for improving ECA systems: • extracting knowledge • markup language for specifying ECA behaviors • …

  12. Overview of the session • Introduction to XML structure • Demos • Use of APML in an ECA system • XML markup in MARY Text-to-speech • Hands-on: The ANVIL tool and the resulting XML markup