Outline

1. Semiotics and semantic: tools for an effective appropriation of information, communication and health technologies Carine Duteil-Mougel, Anne Geslin-Beyaert, Laurent Billonnet, Nicole Pignier, Didier Tsala-Effa, Jean-Michel Dumas University of Limoges, Limoges, France laurent.billonnet@unilim.fr Medetel 2012The international eHealth, Telemedecine and Health ICT forum 18-20 April 2012, Luxembourg

2. Outline Introduction Learning from experimentation I.What about semiotics? II. What about semantics? III. The object - subject interaction IV. The appropriation of Communicant objects V. The Voice User Interfaces Conclusion To the human-machine semiotic&semantic interface (HMSSI)

3. Learning from experimentation… • ICT : a wish but still an unsuitable system; • Feeling of disappointment. • Poor consideration of human being, user friendliness, personalization, • dignity and respect. • Feeling of intrusion and stigmatization. • Training and information needed for a better appropriation of equipments. • Need for an ethical dimension. • Appropriation of ICT tools is a condition of success. • … to the semiotic and semantic approach

4. I. What about semiotics? • Definition • Semiotics is a human science which analyzes systems of signification, whatever the language expressing them: texts, pictures, spaces, etc. • 2. Semiotics of the ICT includes two main directions: • multimedia, multimodal and hypertextual contents; • graphic and material interfaces. • In both cases, the semiotician questions the conditions of perception • of a text or an object on a digital support.

5. I. What about semiotics? • 3. Semiotics of the ICT in the field of health • The way the user, with his current skills and the gestural, cultural experiences he developed in the practices of his everyday life, perceives digital objects intended to help him. • As a consequence, the ICT object is acceptable only insofar as it interacts with the person through his own experiences • (ex: a light path does not fit and tell the story of the person). • The semiotician thinks out the coherence between the design of an ICT device and the perception the user can have of the interaction with this device.

6. II. What about semantics? • 1. Definition: • Semantics is the most recent of the branches of linguistics and is still in the process of being constituted. • We have adopted a semantic approach (developed within the Saussurian tradition) that we consider to be the most sensitive to diversity: interpretative semantics. • By privileging the study of meaning, interpretative semantics focuses on the Cultural Objects, rather than the sign, and defines meaning as interpretation. • 2. Objectives: • To describe a great diversity of Cultural Objects • To specify the incidence of social practices

7. II. What about semantics? • 3. Semantics of ICTs: • As Cultural Objects, ICTs belong to the human sphere (concept of Umwelt formulated by Jakob von Uexküll). This surround is composed of polysemiotics performances (involving many signs systems). • The meaning of these objects is then studied in relation with the actions to which they are associated. • The study of users’ discourses is very interesting from this point of view, and should lead to formulate recommendations for a more efficient appropriation of the ICTs, and a real implication of the persons.

8. III. The object - subject interaction • The user (subject) establishes a junction with the technological object: the relationship is called a “con-junction”. • Referring to the three levels stake in social practices (presentational, semiotic, physical), the object could become a mediator between the user and its environment. Presentationallevel Ex. mental representations Semiotic mediation Social practice Ex. speech and gesture (nonverbal communication) Semioticlevel Physicallevel Ex. movement The term of “Semiotic mediation” is thusly used

9. IV. The appropriation of Communicant objects • In terms of semiotics of anthropic zones, the appropriation expresses the way to go from the distal zone to the practical proximal zone. Distal Zone (strangeness) Proximal Zone (adjacency) Identity Zone (coincidence) The object which has the FETISH status is seen as "familiar" and that make easier its acceptation and its daily use. FETISH Object isseen as familiar IDOL Object isseen as technical The problematic is to tip the ICT objects from IDOL to FETISH status. • Our Hypothesis: the Voice User Interfaces (VUI) which have recourses to spoken language, make easier the pass from IDOL to FETISH, in comparison to the Graphical User Interfaces (GUI). Thirdperson: He/She/It/ That PAST and FUTURE Elsewhere Second person: You NEAR PAST and FUTURE There First person: I NOW Here

10. V. The Voice User Interfaces • The Recognition of Continuously Spoken Sentences allows the user to address oneself from a natural way to the system without any preliminary specific learning to use the system. • The recourse of the natural dialogue(unlimited) begs the interrogation about the semantic interpretation, and underlines the role of the situational context. • Objective: Predict functional Voice User Interfaces which take into account the expectation and the use of languages (lexicon, syntax, semantics, pragmatics, etc.)

11. Conclusion To the human-machine semiotic&semantic interface (HMSSI) • Based on the vocal and/or the gesture approaches • Several mono or multimodal interfaces techniques • dematerialized interfaces • vocal interfaces (analysis and synthesis) • virtual interfaces using gesture recognition • augmented reality and virtual assistive information • haptic, visual or auditive augmentation as a an assistance • haptic, visual or auditive augmentation as a validation • intuitive tactile sensitive processes • as a projection of an intuitive gesture

12. Some more information?… • Contact • Laurent Billonnet laurent.billonnet@unilim.fr Professor at the University of Limoges ENSIL engineering School Director of the BSc « Home Automation for Elderly and Disabled People »