1 / 39

A Framework for Perceptive, Attentive, Cognitive, and Social-Interaction Enhancement From User-Centred to Ego-Centr

A Framework for Perceptive, Attentive, Cognitive, and Social-Interaction Enhancement From User-Centred to Ego-Centred Design Morris Goldberg. Attention. Perception. Cognition. PACSIE. Social Interaction. Sreekar Krishna Prof. Panchanathan.

napoleon
Download Presentation

A Framework for Perceptive, Attentive, Cognitive, and Social-Interaction Enhancement From User-Centred to Ego-Centr

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Framework • for • Perceptive, Attentive, Cognitive, • and • Social-Interaction Enhancement • From User-Centred • to • Ego-Centred Design • Morris Goldberg

  2. Attention Perception Cognition PACSIE Social Interaction Sreekar Krishna Prof. Panchanathan

  3. Perception, Attention, Cognition, Social-Interaction • Perception: • Hear a baby cry. • Attention: • Listen so as to hear one’s baby cry • Cognition: • Identify the reason for crying • Social-Interaction: • Rock the baby to stop the baby from crying Wednesday, June 4, 2014 PACSIE: Ego-Centred Design 3

  4. Behaviour Responses:Movement and Autonomic • •Movement: • Turn head to better hear the baby cry • Move towards the baby, inspect baby • Pickup baby, rock the baby • Autonomic: • Posture: as baby may cry • Heart-rate: if baby cries Wednesday, June 4, 2014 PACSIE: Ego-Centred Design 4

  5. Enhancement • Applications • Cerebral: injury, disease, and aging • Autism, prosopagnosia • « Extra eyes » • Training, drugs, invasive • Practice makes perfect,… • Assistive device • Non-invasive, customized, adaptive, lifelong Wednesday, June 4, 2014 PACSIE: Ego-Centred Design 5

  6. Motivation: The Carpenter Who Lost His Tools • Visual agnosia for tools • sees tool but lost knowledge of tool • name, function, memory • Haptic tool recognition • feels tool, knowledge recovered • Infancy: body(hands) calibrates vision • Maturity: • vision: dominates • body:mostly below conscious awareness Wednesday, June 4, 2014 PACSIE: Ego-Centred Design 6

  7. PatientDF: Visual form agnosia • CO poisoning heater • « Blind » to many objects • Recognized her mother through voice • Recognized colours, texture • Name some surafces • Red plastic, shiny metal • Spatial acuity intact • Gratings, but not directions • Basic problem recognizing shapes • MA Goodale, AD Milner, Oxford Univ. Press • Sight Unseen: An Exploration of Conscious and Unconscious Vision, 2004 Wednesday, June 4, 2014 PACSIE: Ego-Centred Design 7

  8. Patient DF: Tests • Object recognition (haptic) • Copying objects • Yellow blind to orientation • Grasped any orientation Wednesday, June 4, 2014 PACSIE: Ego-Centred Design 8

  9. Visual processing streams: what, where, how  Wednesday, June 4, 2014 PACSIE: Ego-Centred Design 9

  10. Surrogates: The visually impaired • Visual agnositics • rare, encumbered, difficult, non-stable • The blind • visualize objects by touch, stable, intelligent • Reader • NSF, focus groups Wednesday, June 4, 2014 PACSIE: Ego-Centred Design 10

  11. Haptic object visualisation • Vibro-tactile glove • Sensory substitution: distal visual to proximal haptic • Haptic rendering not possible • Observe haptic manipulation • Uncover distinguishing haptic features/descriptors • Deliver features by haptic cueing • Supervised user learning: haptic cues • Object visualization: user imagination Kahol K., Panchanathan S., “Neurocognitively inspired haptic user interfaces”, Springer J. of MM Tools and App, v 3, pp 15-38 2007. Kahol K, Smith M., “Neuro-cognitively Designed Dynamic Simulations For Laparoscopic Surgical Skills”. MODSIM. Virginia Beach, 2007. pp. 34-39. Wednesday, June 4, 2014 PACSIE: Ego-Centred Design 11

  12. User-Centred Design • Design methodology • Participatory design • focus groups -scenario building • Prototypic User • Average user • Learning • Designer discovery • usage patterns, blog, competition.. • User discovery • short-cuts, hidden functionality Wednesday, June 4, 2014 PACSIE: Ego-Centred Design 12

  13. Framework • Assistive device • white cane.. extension of self…peripersonal space • left-neglect…stroke…line bisection..finger, laser-pointer, ,stick • Haptic exploration • haptic glove 6 blind people pattern varies • depends on degree and onset of blindness, other factors • Ego-centred design • understanding of the individual • perceptive, attentional, cognitive, social (PACS) Wednesday, June 4, 2014 PACSIE: Ego-Centred Design 13

  14. Ego-Centred Design: Principles • 1st Principle: Sensory-Motor Coupling • 2nd Principle: Polymodal Reinforcement • Edelman, GM, « Neural Darwinisim: The Theory of Neuronal Group Selection », 1989. • 3rd Principle: Graft Onto Existing Scaffolds • Thelen and Smith,“A Dynamic Systems Approach to the Development of Cognition and Action”, 1996. • 4th Principle: Self Evaluation Wednesday, June 4, 2014 PACSIE: Ego-Centred Design 14

  15. 1st Principle: Sensory-Motor Coupling • Active sensing • sensory adaptation to constant signal • adapt to pressure on body, smell, even vision • shift body, sniff, eye saccade, …. • « What » is being sensed? • behavior is visible • deduce « what » • haptic manipulation- descriptors • Haptic rendering • motor-sensory coupling • individual moves, assistive devices provides stimuli Wednesday, June 4, 2014 PACSIE: Ego-Centred Design 15

  16. 2nd Principle: Polymodal Reinforcement • Modes • vision: colour, shape, orientation, motion…. • somatosensory: light and heavy pressure, pain, vibration… • auditory, proprioceptive, olfactory:……. • Existence-pathological cases • Agnosia: loss of mode/function • Synaesthesia: extra function-e.g see sound, hear color Wednesday, June 4, 2014 PACSIE: Ego-Centred Design 16

  17. 2nd Principle: Polymodal Reinforcement • Examples • texture: somatosensory, visual, and auditory • taste: smell and somatosensory(tongue) • Associative memory • Bartlett: memory is a re-creative process • Move hands, visualize, hear , feel Wednesday, June 4, 2014 PACSIE: Ego-Centred Design 17

  18. 2nd Principle: Polymodal Reinforcement • Exploitation • Wagner: musical identities • rehabilitation: colour code objects, audio prompts • Haptic glove • audio and haptic cues for textures Wednesday, June 4, 2014 PACSIE: Ego-Centred Design 18

  19. 3rd Principle: Graft Onto Existing Scaffolds • Child development • reaching: months of effort, muscles, vision proprioceptive • orienting to people: instintive attraction to face-like patterns • Continuum • scaffolding: basis for new functionalities • development nevers stops • Blind and body language • auditory cues: nervous movement, breathing, voice inflection • assistive device: correlate/integrate Wednesday, June 4, 2014 PACSIE: Ego-Centred Design 19

  20. 4th Principle: Self Evaluation • Gait varies • load, terrain, physical condition • trial and error: self evaluation (pain, comfort) • Continuous self-evaluation • Innate Behavioral Responses • attraction to eye-like objects • move eyes- get reward (smile,hug)- • emotional-based value system Wednesday, June 4, 2014 PACSIE: Ego-Centred Design 20

  21. What do we know about them? Everyday Social Interactions Wednesday, June 4, 2014 PACSIE: Ego-Centred Design 21

  22. Social Interactions: Complex Visuo-Motor Interplay Sensory Cognitive Eye Contact Social Interactions Intent to interact Perceptual Motor Move towards each other (Keep in mind Proxemics) Shake hands Handshake Conversation distance Wednesday, June 4, 2014 PACSIE: Ego-Centred Design 22

  23. Psychological Basis of Social Interactions • Emotions • Interpersonal Attributes • Dominance • Trust • Like/Dislike • Personality • Communication • Turn Taking • Feedback • Attention • Iconic Cues, like Victory or Stop hand gestures. • Intrinsic Cues, like moving close to someone Behavioral Aspects Communication Aspects M. Argyle, The Psychology of Interpersonal Behaviour, Penguin UK, 1999.   PACSIE: Ego-Centred Design

  24. Psychological Aspects – Dyadic Interactions Expressions Body Mannerisms Communicative Gestures Feel Send Receive Feel A. Teeters, R.E. Kaliouby, and R. Picard, “Self-Cam: feedback from what would be your social partner,” ACM SIGGRAPH 2006 Research posters, Boston, Massachusetts: ACM, 2006, p. 138. M.G. Almore, “Dyadic Communication,” The American Journal of Nursing, vol. 79, Jun. 1979, pp. 1076-1078.   PACSIE: Ego-Centred Design

  25. Insula and the Mirror Neurons Mirror Neurons Insula Limbic System Simulate Facial Expression Feel the Emotion L. Carr, M. Iacoboni, M. Dubeau, J.C. Mazziotta, and G.L. Lenzi, “Neural mechanisms of empathy in humans: A relay from neural systems for imitation to limbic areas,” Proceedings of the National Academy of Sciences of the United States of America, vol. 100, Apr. 2003, pp. 5497-5502.   • Derived out of fMRI studies on human viewing facial images • Enhanced activity in the Mirror Neuron Area, Insula and Limbic System observed • Findings are just anatomical patterns • Exact Pathway is still not discovered PACSIE: Ego-Centred Design

  26. Bringing it all together Everyday Social Interactions – The PACSIE Model PACSIE: Ego-Centred Design

  27. Principle 1: Sensory-Motor Coupling • Social Interactions are integration of complex sensory and motor actuations and responses. Visual Engagement Perception Coordinated Motor Interaction – Eye Gaze, Head Nod Attention Cognition Communication – Asynchronous Exchange Social Interaction Complex Social Engagement Enactor (Encoding) Recipient (Decoding) PACSIE: Ego-Centred Design

  28. Principle 2: Polymodal Reinforcement • Social Interactions are integration of multichannel information(from the enactor and the recipient’s perspective). Recipient (Decoding) Enactor (Encoding) Face 27% Visual Body Non-verbal 65% 19% Voice 18% Audio Speech 35% 35% Verbal N. Ambady and R. Rosenthal, “Thin slices of expressive behavior as predictors of interpersonal consequences: A meta-analysis.,” Psychological Bulletin. Vol. 111(2), vol. 111, Mar. 1992, pp. 256-274.   PACSIE: Ego-Centred Design

  29. Principle 3: Use existing skills (i.e. Scaffolding) • Social Interactions capabilities are very individualistic based on when the person becomes blind. Social Skills Social assistance need varies depending on when the person loses his/her sight. Birth Developmental Years Professional Years Geriatric Years Congenitally Blind Adventitiously Blind PACSIE: Ego-Centred Design

  30. Principle 4: Recalibration by Self Evaluation • Social Mirroring is the process by which effective social interactions are learned and propagated. Perception Action Mirror Neurons Social Mirror Hello !!! Welcome to CUbiC Self Evaluate M. Iacoboni, Mirroring People: The Science of Empathy and How We Connect with Others, Picador, 2009.   PACSIE: Ego-Centred Design

  31. Body Rocking Condition Social Solutions Through PACSIE PACSIE: Ego-Centred Design

  32. Role of vision Learning through observation Feedback on social mannerisms Visual Impairment affects both aspects of social interactions PACSIE: Ego-Centred Design

  33. Stereotypic Mannerisms Head- 13.1% Though body rocking is not the most occurring mannerism, it is the most noticed in a social scene Face - 3.4% Hand - 60.3% Body - 6.9% Trunk - 10% Leg - 6.4% V.J. Eichel, “Mannerisms of the blind: A review of the literature,” Journal of Visual Impairment & Blindness, vol. 72, 1978, pp. 125-130.   PACSIE: Ego-Centred Design

  34. Body Rocking in Adults • Rocking is a natural development behavior of the cerebellum’s balance system • Body rocking in adults is due to the lack of social feedback PACSIE: Ego-Centred Design

  35. Solutions Focus on Controlling the problem and not curing it !!! Self Monitoring Intervention • Immediate feedback • Does not address the problem • Reduces the symptom • Chance of recurrence is high • Replace with Cognitive Task • Provides for symptom substitution • Chance of recurrence is low R.L. Loftin, S.L. Odom, and J.F. Lantz, “Social Interaction and Repetitive Motor Behaviors.,” Journal of Autism & Developmental Disorders, vol. 38, Jul. 2008, pp. 1124-1135. M. Celeste, “Social Skills Intervention for a Child Who Is Blind,” Journal of Visual Impairment & Blindness, vol. 101, 2007, pp. 521-533.   PACSIE: Ego-Centred Design

  36. Social Solution PACSIE Social Feedback Body Rocking Social Solution Control Self Evaluate PACSIE: Ego-Centred Design

  37. Social Interaction Assistant: Work-inProgress Wednesday, June 4, 2014 PACSIE: Ego-Centred Design 37

  38. Person, Gaze, Expression, Identity Detector Wednesday, June 4, 2014 PACSIE: Ego-Centred Design 38

  39. Ego-Centred Design: Principles • 1st Principle: Sensory-Motor Coupling • Head movement controls sensing(far-near,left-right) • 2nd Principle: Polymodal Reinforcement • vibration and sound • 3rd Principle: Graft Onto Existing Scaffolds • mood recognition, voice recognition, footstep recognition.. • 4th Principle: Self Evaluation • •user evaluates/calibrates the signals from the SIA • •eventually SIA will also self-evaluate Wednesday, June 4, 2014 PACSIE: Ego-Centred Design 39

More Related