Y.Tabata 1 , T.Kuroda 2 and K.Okamoto 2 1 Kyoto College of Medical Science

1. ICDVRAT2012 Development of a glove-type input device with the minimum number of sensors for Japanese finger spelling Y.Tabata1, T.Kuroda2 and K.Okamoto2 1Kyoto College of Medical Science 2Kyoto University Hospital

2. Communication barrier of the hearing impaired The hearing impaired have the communication barrier in their social lives. Communication Different grammar E-mail Writing image Sign Language sign language is the most familiar

3. To overcome the communication barrier… Many researchers have tried to develop a sign translation system. Signer Translation Hello Hello Hi … subtitle Sign Animation Necessary to have input device to capture sign

4. Glove-type input device • Glove-type input device is … • essential device for VR • used to obtain hand postures • expensive • Price of the device would be higher than the price of a car.. Still expensive as the price of the glove available for sign capturing

5. Proposed method to define the minimum number of sensors • Proposed the method to define the minimum number of sensors to capture a set of the given hand posture ( in ICDVRAT2010 ) • Estimated that a data-glove equipped with six low-resolution sensors could measure hand postures expressed in Japanese Finger spelling of Japanese Sign language. Possibility to decide a number of sensors for user’s demands

6. Objective • To make a prototype of data-glove with six sensors for Japanese finger spelling. • To evaluate the prototype Prototype

7. Hand postures of Japanese Finger spelling • Japanese Finger spelling • Alphabet of Japanese Sign Language

8. Definition of minimum number of sensors to capture Creating code-sensor Relation Table 2. Seeing if proposed optimizing problem reduces to a set cover problem 3. Trying to get solutions by using set cover algorithm Y.Tabata, T.Kuroda, K.Okamoto, Proc. ICDVRAT2010

9. Detail of proposed method Creating code-sensor Relation Table Code:α Code:β Code:γ Fully furnished data-glove Given hand postures Code-sensor relation table

10. Detail of proposed method 2. Seeing if proposed optimizing problem reduces to a set cover problem Code-sensor relation table( with simple switches) • All the code are identified with simple switches proposed problem reduces to a set cover problem • All the code are notidentified with simple switches proposed problem selects a brute-force method Pairwise code-sensor relation table

11. Detail of proposed method(4/4) 3. Trying to get solutions by using set cover algorithm • Omit non-contributing sensors and replaceable sensors from the instance ( pairwise code-sensor relation table ) . • Solve the instance by using the exponential time algorithms of set cover problem Pairwise code-sensor relation table V17 V26 V27 U V 2 sensors 3 combinations : （V17 , V26） （V17 V27）（V26 V27 ） minimum number of sensor

12. The obtained set cover for Japanese Finger spelling • Minimum number of sensor : six sensors • Five bending sensors ( yellow ) • one contact sensor (red) A set cover for Japanese Finger spelling Sensor Layout

13. Prototype for Japanese finger spelling • Prototype • Is applying to the existing StrinGloveTM • Consists of five two-bits bend sensors and one one-bit contact sensor Prototype

14. Evaluation • The prototype is evaluated • Subject expressed the 28 hand postures of Japanese finger spelling by the prototype. • Total number of subjects • three (female : two, male : one) the 28 hand postures of Japanese finger spelling

15. Results • The prototype distinguished about 82 % of the given hand postures. • The main cause of errors were the two sensors, no.5 and no.6.

16. Discussions • Sensor no.6 :one of the main cause of errors • A contact sensor used for measuring the contact between middle and ring fingers. • A proximity sensor was utilized as the contact sensor and the sensor sometimes misfires. • Even when the distance between two fingers is not zero. • Sensor ‘s size is large It may be useful to use sensor instead of the proximity sensor and to downsize the sensor

17. Discussions • Sensor no.5 :one of the main cause of errors • No.5 is the bending sensor for measuring angle of DIJ of pinkie. • The slip between glove and pinkie might cause a lot of error. • It may clear that • a single sensor measurement error becomes fatal error under the optimized data-glove. more stretchable material to fit the glove tightly

18. Conclusions • We made a prototype of data-glove for Japanese finger spelling. • The prototype is a data-glove with six sensors. • The prototype was evaluated. • The prototype distinguished about 82 % . • The prototype failed to distinguish the given hand postures because of simple error of sensors. • It would be necessary to improve some sensors.

19. Thank you very much for your kind attention.

20. ICDVRAT2012 Development of a glove-type input device with the minimum number of sensors for Japanese finger spelling Y.Tabata1, T.Kuroda2 and K.Okamoto2 1Kyoto College of Medical Science 2Kyoto University Hospital