1 / 11

Affective Interfaces Present and Future Challenges

Affective Interfaces Present and Future Challenges. Introductory statement by Antonio Camurri (Univ of Genoa) Marc Leman (Univ of Gent) MEGA IST-20410 Multisensory Expressive Gesture Applications. Introductory Statement (1/3).

zazu
Download Presentation

Affective Interfaces Present and Future Challenges

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Affective InterfacesPresent and Future Challenges Introductory statement by Antonio Camurri (Univ of Genoa) Marc Leman (Univ of Gent) MEGA IST-20410 Multisensory Expressive Gesture Applications

  2. Introductory Statement (1/3) • KANSEI, Affect, Emotion, Personality, Expressiveness, Gesture, . . . • Psychology, AI, CHI, Art and Humanities • Interfaces: • Enhance interfaces by means of non-verbal expressive data modeling and communication • Embed models of emotion and personality in natural language dialog (e.g., web agents, . . .)

  3. ...Introductory Statement… (2/3) • (USA) Affective Computing: Bates/CMU, B.Hayes-Roth/Stanford, Picard/MIT, ... O’Rorke-Ortony model • Games/entertainment, Web-agents with personality, wearables, mobile devices... • (Japan) KANSEI Information Processing: S.Hashimoto/Waseda, K.Mase/ATR, ... • KANSEI Industry (Art, Entertainment, Fashion, Cosmetics, Kansei-Bots, ...); the HUMANOID Project (e.g., elderly assistance by “Kansei machines/interfaces”)

  4. ...Introductory Statement (3/3) • An example - a spectator observes a “microdance”: • Which are the lower-level “cues” in movement which may explain “expressiveness”? (eg, rigid/fluid, direct/flexible, extrovert, energy, rhythm features…) • does the spectator understand the dancer’s intended emotional expression ? (eg, “sad”, “angry”) • “KANSEI” or “sensible” content in the microdance: how much intensely the spectator is “hit” by an inspired dancer performance ? (similar to the “arousal” problem in emotion psychology theories) • Music, Sound and Voice • Expressive spaces as a new perspective on multimodality

  5. MEGA IST-40210: Toward an Original European Route • MEGA (EU IST-20410 www.megaproject.org) aims at consolidating a third original European route, gounded on: • art and humanistic theories of non-verbal communication • “expressive gestures”: expressive cues, emotion, “sensible”/KANSEI processing • Focus: non-verbal expressive gestures communication in artistic scenarios, trying to inprove technology by arts and humanities

  6. The notion of “Gesture” • non-verbal epistemic unit: e.g., movements of the human body, particular occurrences in music • defined by criteria related to human information processing, in particular in motor and perceptual actions • has typical beginning and ending • describes trajectories in a feature space • may occur at different levels:e.g., in music, an individual note may be considered as a gesture, with a given attack, sustain and decay pattern. Also a sequence of notes may be considered a gesture,or a repeating rhythm pattern.

  7. “Expressive Gesture” • A gesture, in principle, may be neutral, hence operating as a self-contained unit, yet many gestures are called ‘expressive’ • Expressive gestures refer to a semantics outside the gesture pattern. This semantic may relate to synaesthetic (e.g. sound-colour), kinaesthetic (sound-movement, cf. Laban), and cenaesthetic issues (sound-sensation-emotion) • The notion of gesture as a fundamental category of human action and perception, that ultimately relates to affective and sensitive computing

  8. “Gesture” in MEGA • Definition of the notion of gesture in multimodal artistic and cultural scenarios • Intermodality or mapping of gestures from one to another domain • Recognition of features that are relevant for expressive gestures, in particular also for affective and sensitive computing

  9. MEGA: Research • Analysis of expressive gesture in • movement (incl.dance) WP3 • audio (incl. music) WP4 • Synthesis of expressive gesture in • movement and visual media WP5 • sound (incl. music) WP6 • Multimodal Integration • Mapping strategies from input modalities to output media • Open software platform (EyesWeb) to support research and to enable commercial exploitations

  10. MEGA: open architecture • Based on EyesWeb (DIST), • open platform supporting MEGA research and apps • the platform integrates “modules” for expressive analysis, synthesis, and mapping strategies. • visual language, Wizard for plug-in’s devel. • based on industrial standards (MS COM/DCOM) • engineered release, coming from previous versions used in concrete apps • EyesWeb user community • Integrated with other tools available in MEGA: IPEM ToolBox (IPEM), DOVRE (Telenor), ...

  11. MEGA: Applications • Exploitation: • Art and theatre: MEGA consortium • Musical instruments: GeneralMusic • Visual “soundtrack”: Eidomedia • Distributed cooperative apps: Telenor • Demos at IBC2001? • Public artistic events planned: music theatre, interactive museum exhibits

More Related