Coding Scheme in Gestures Analysis - PowerPoint PPT Presentation

Coding Scheme in Gestures Analysis

Liang Zhou

Dr. Manolya Kwa

• Gesture is the special motion(s) which express specific meaning from thought. ( Another form of Speech)
• G-Analysis is the studying of human behaviour which purpose to understand the human actions and their thought.

• Gesture analysis by computer becomes an important role in science studying field. (Sign language)
• it is key role in human and computer interaction research
• It has significant scientific value.

• Coding Gestures is the process of translating attributes or properties of the gestures (i.e Finger Form) into a form that

researchers can systematically analyse.

• Coding gesture is the important to
• HCI need coding gesture. (One of the most important concepts is that of direct manipulation)

In Cognitive studying, the abstract concept

of coding scheme can’t be implement in HCI.

Reason:

• Abstract Code
• Ambiguous Expression.

• Physical Actions:

L-Action, M-Action, D-Action

• Perceptual Action
• Functional Action
• Conceptual Action: Goals etc.

• 1. Speech and Gesture are the same system.
• 2. Interpret the thought more clearly than the sentence and words (Better Understanding)
• 3. Gesture express the thought instantly but speech systematically .
• 4. Gesture Recognition Tools are mature and widely use in Gesture Analysis.

• How to prove it?
• How to Code the gesture ?
• How to improve the reliability of Coding Scheme?
• How to get the sufficient data ?

• 4 participants : 2 task
• Task 1: Cognitive Progress
• Task 2: Gesture Capture Phase.
• Cognitive progress : Sketching the chair and record the whole phase.
• Capture the gesture and motion by 2 cameras. One for body motion, one for Hand motion.

• Sketching Record

2. Gesture Image.

• Motion::
• attributes: M_Name, Hand_type frequency(default) TimeStamp Position Direction FingerForm
• M_Name: {}
• Hand_type: {LH,RH,TH}
• frequency: {Single, Repeated }
• TimeStamp: {ST,FT}
• Position: {Left, Lower-Left, Upper-Left, Center, Right, Lower-Right, Upper-Right, Lower}
• Direction {(1,0), (1,1),(-1,0),….}

• Gesture ::
• Attributes : Name, TimeGap, PosSS, Objects
• Name: (GestureName)
• TimeGap: (Time Second)
• PosSS: (Distance)
• Objects( parts of chair)

• The Anvil could compute statistics on an annotated video sequence and the annotation element which we defined in the coding scheme will be represented the same behavior and grouped together. Then, we will understand the label between Gestures and Objects. Basic on the Kawali‘s experiment result, we will understand that what kind of the gestures is mapping to which behaviors and the gestures is mapping to the goal /goals or not.

• We have found out 6 common gestures
• However, because of lack of the data , we just can simply to capture and represent it by our coding scheme.
• But we’ve already find that some data in cognitive actions show a strong correlation with gestures.

• One of the most occurring gesture:
• “This part of the chair I think is too small or big.”
• The hand focus on the size of the part they drew. The level of the size and amount should be expressed..
• P1, P3, P4, P5

• We provide a new methodology to understand the cognitive actions and provide its possibility to implement HCI in design field
• Future work :
• 1. Coding scheme need to cover more other’s field. For example , facial and Head movement.
• 2. More task in the experiment will gain sufficient data to analysis and explore the gesture.

Question?