Real Time Facial Expression Recognition and Imitation

1. International Journal of Trend in Scientific Research and Development (IJTSRD) Volume 4 Issue 4, June 2020 Available Online: www.ijtsrd.com e-ISSN: 2456 – 6470 Real Time Facial Expression Recognition and Imitation Varsha Kushwah, Madhuri Diwakar, Tej Kumar, Dushyant Singh Department of Electronics & Communication Engineering, R B S Engineering Technical Campus, Bichpuri, Uttar Pradesh, India ABSTRACT The object of this paper was Real Time Facial Expression Recognition (FER) has become main area of interest due to its wide applications. Automatic Facial expression recognition has drawn the attention of researchers as it has many applications. Facial Expression Recognition gives important information about emotions of a human being. Many feature selection methods have been developed for identification of expressions from still images and real time videos. This work gives a detailed review of research works done in the field of facial expression identification and various methodologies implemented for facial expression recognition. How to cite this paper: Varsha Kushwah | Madhuri Diwakar | Tej Kumar | Dushyant Singh "Real Time Facial Expression Recognition and Imitation" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-4 | Issue-4, June 2020, pp.1522- 1524, URL: www.ijtsrd.com/papers/ijtsrd31584.pdf Copyright © 2020 by author(s) and International Journal of Trend in Scientific Research and Development Journal. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (CC (http://creativecommons.org/licenses/by /4.0) IJTSRD31584 BY 4.0) INTRODUCTION: Our project titled as "Real Time Facial Expression Recognition and Imitation" has been undertaken with the aim of estimation of facial expression guiding by a program for Raspberry Pi to evaluate a person's exact expression or expression type. The technique we used for expression type estimation contains two main component-type estimation divides the types only into rough groups (e.g., angry, disgust, fear, happy, sad, surprise, neutral). Studies on human facial emotion recognition have revealed that computing models based on regression modeling can produce applicable performance. Emotion detection and recognition were introduced by researches with human observers. Automatic recognition and the study of the facial emotional status represent substantial suggestions for the way in which a person performs, and these are very helpful for detecting, inspection and keeping safe vulnerable persons such as patients who experience mental issues, persons who endure significant mental pressure, and children with less ability to control themselves. With emotion recognition ability, machines such as computers, robots, toys and game consoles will have the capability to perform in such a way as to influence the user in adaptive ways relevant for the client’s mental condition. This is the key knowledge in recently proposed new ideas such as emotional computers, emotion-sensing smart phones and emotional robots. Most studies have focused on emotional symptoms in facial expressions. In recent years, scientists have studied effective transition through body language. The recognition of entire body gestures is expressively more diff icult, as the shape of the human body has more points of freedom than the face, and its overall outline differs muscularly through representative motion. Most face detection algorithms are designed in the software field and have a high detection rate, but they often needs several seconds to detect faces in a single image, a processing speed that is insufficient for real time applications. To overcome this problem we used an easy hardware implementation of Raspberry Pi which is the fastest method of face detection and estimation of facial expression accurately. For the extraction of facial feature of human we used camera pi which is an excellent add-on for Raspberry Pi Processor to take pictures and record videos with the possibility to apply a considerable range of configurations and effects. An face emotion recognition system comprises of two step process i.e. face detection (bounded face) in image followed by emotion detection on the detected bounded face. The following two techniques are used for respective mentioned tasks in face recognition system. 1.Haar feature-based cascade classifiers : It detects frontal face in an image well. It is real time and faster in comparison to other face detector. This blog-post uses an implementation from Open-CV. 2.Xception CNN Model : We will train a classification CNN model architecture which takes bounded face (48*48 pixels) as input and predicts probabilities of 7 emotions in the output layer. @ IJTSRD | Unique Paper ID – IJTSRD31584 | Volume – 4 | Issue – 4 | May-June 2020 Page 1522

2. International Journal of Trend in Scientific Research and Development (IJTSRD) @ www.ijtsrd.com eISSN: 2456-6470 In our day to day life emotions or facial expression are the prime factor which are required for communication purpose. For humans it is quite easy to detect an emotion but difficult for a computer or a machine to do so. Human emotions are mainly classified into 7 emotions i.e. Neutral, Happy, Sad, Anger, Disgust, Fear and Surprise. This project deals with 4 emotions they are Neutral, Happy, Sad and Surprise. As we are developing the need and importance of automatic emotion recognition has increased which supports Human Computer Interaction applications. Facial expression defines the emotions of an individual which is required for Human Computer Interaction (HCI) in this project. Apart from Human Computer Interaction the system could be used for monitoring medical patients emotional states and stress levels. Emotion recognition can be done by text, vocal, verbal and facial expression. In 1968, Albert Mehrabian pointed out that in human to human interaction 7% of communication is contributed by verbal cues, 38% is contributed by vocal cues and major portion 55% is contributed by facial expressions. Fig1: Real Time Emotion Recognition System Fig1: Real- time Emotion recognition system Raspberry Pi II is a credit card sized computer which has system on a chip Broadcom BCM2835. It contains an ARM1176JZFS, with floating point, running at 900 MHz and a video core 4 GPU. The architecture of proposed system is shown in Fig. 1 and explained as follows: The input image in real time is captured through webcam and fed to emotion recognition software as input. Emotion recognition software is deployed in the Raspberry Pi II, which gives classified emotion as output. The recognized emotion is displayed in the monitor. The operations performed by the software deployed in the Raspberry Pi II is shown in Fig. 2. The algorithm for real time implementation of emotion recognition using Raspberry Pi II is explained as follows Step 1: Input image is captured through webcam. Step 2: Viola-Jones [6] face detection technique is used to detect the facial image. Viola-Jones used Haar wavelet concept to develop integral image to detect face. Haar features consider the different intensity of values of adjacent rectangular region as different area of face has different value of intensity from other region. After detection, facial image is saved for further processing and non-face area is removed. Step 3: In image preprocessing, image is cropped according to required size and converted in gray image. This cropped image is used as input to Sobel filter for smoothing to remove the noise. Step 4: Feature extraction is based on geometric approach for which Active Shape Model (ASM) is used. ASM automatic fiducial point location algorithm is applied first to a facial expression image, and then Euclidean distances between center gravity coordinate and the annotated fiducial points coordinates of the face image are calculated. In order to extract the discriminate deformable geometric information, the system extracts the geometric deformation difference features between a person's neutral expression and the other basic expressions. In ASM input face shape is iteratively deformed to get the shape model. After comparison with shape model feature point of input facial image is extracted. Methodology - In the proposed method, the objective is to develop real time emotion recognition from facial images to recognize basic emotions like anger, disgust, happiness, surprise and neutral. We have used CMU Multiple database, which is a collection of images from 337 subjects with a variety of different facial expressions including neutral, happiness, surprise, disgust and anger. The subjects include 235 males and 102 females with different level of illuminations and poses. Viola-jones face detection method for face detection, Active shape Model (ASM) for extracting facial points and AdaBoost classifier have been used for developing the emotion recognition software. @ IJTSRD | Unique Paper ID – IJTSRD31584 | Volume – 4 | Issue – 4 | May-June 2020 Page 1523

3. International Journal of Trend in Scientific Research and Development (IJTSRD) @ www.ijtsrd.com eISSN: 2456-6470 To train the model, frontal images of 5 emotions, 60 subjects from CMU MultiPIE database is used where sample images from the database are shown in Fig 4. Feature points are marked on these images as shown in Fig 5. These points are then normalized and formed as a feature vector. This operation is repeated for all the subjects and emotions, and the feature points forms a single feature vector which is given to the classifier for training. techniques. The feature extraction ways area unit exploited together with comparison. in line with the survey varied approaches area unit out there for establish face expression. during this paper numerous face expression recognition techniques, its associated areas and numerous analysis challenges is being surveyed. Methods with higher recognition rate have sensible performance These techniques give a temporal answer to the face expression recognition downside. feeling detection from face expression is a problem that causes difficulties. So, analysis during this space can continue for years as several solutions needs to be to make a perfect programmed. A comparative study is completed with numerous image information bases by differentiating the options. References [1]Tuark MA, Peintland AP. feeling recognition mistreatment eigenfaces. In: laptop Vision and Pattern Recognition. Proceedings,IEEE Conference on. IEEE; 1991, p. 586-91. [2]M. El Ayiadi, M. S. Kaamel, and F. Kaerray, “Survey on feeling recognition: options, classification schemes, and information,” Pattern Recognit., vol. 44, no. 3, pp. 572– 587, 2011. [3]Y. Leicun, Y. Beingeo, and G. Henton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, 2015. [4]J. Schmedhuber, “Deep Learning in convolutional neural networks: an summary,” Neural Networks, vol. 61, pp. 85–117, 2015. [5]J. Nagiam, A. Khoesla, M. Kaim, J. Naim, H. Leei, and A. Y. Nig, “Multimodal Aeriel Deep Learning,” Proc. 28th Int. Conf. Mach. Learn., pp. 689–696, 2011. [6]F. Deipl and T. Voigt, “Automatic feeling recognition from Expression,” 2010. [7]S. Lugovec, I. Dinder, and M. Hoervat, “Approaches and applications of feeling recognition,” 2016 thirty ninth Int. Conv. Inf. Commun. Microelectron. MIPRO 2016 - [8]B. Schuller, G. Rigoll, and M. Lang, “Emotion recognition combining Physical options and linguistic info Proc., no. November 2017, pp. 1278– 1283, 2016. during a hybrid support vector machine - belief specification,” Acoust. Speech, Signal method., vol. 1, pp. 577–580, 2004. [9]C. Bussieo ett ail., “IEMOCAP: Interactive emotional dynamic motion capture information,” Laing. Resoer. Evail., vol. 42, no. 4, pp. 335–359, 2008. [10]J. H. Haensen, S. E. Boiu-Ghaizale, R. Sarikaaya, and B. Pelliom, “Getting Started with SUSAS: A Speech underneath Simulated and Actual Stress information,” Eurospeech, pp. 1743–46, 1997. [11]K.Meisser; J.Mataes; J.Keittler;K., “XM2VTSDB: The Extended M2VTS information.” [12]I. Sneiddon, M. Mc.Rorrie, G. MicKeown, and J. Hanratty, “The iatrogenic natural feeling information for detection,” IEEE Trans. Affect. Comput., vol. 3, no. 1, pp. 32–41, 2012. [13]M. Slaeney, “BabyEars: A recognition system for emotive vocalizations,” Speech Commun., vol. 39, no. 3– 4, pp. 367–384, 2003. [14]K. D. Gairett et al., “Correlates within the Comprehension of Emotional Prosody,” no. February, p. 19104, 2002. [15]E. Douglaas-cowiee, R. Coweie, and M. Schröoder, “A New feeling Database: concerns, Sources and Scope,” In, pp. 39–44, 2000. laptop Society Fig 5: Fiducial points extracted for CMU MULTIPIE database Raspberry Pi: The main hardware used in this project is Raspberry Pi 3 model B. It is a Linux based platform which uses Python as programming language. In Linux software development is quite simple as it is an open source code development environment. Technol. Electron. Camera Module: The camera module used in this project is 8MP high definition camera for better picture quality. Conclusion Facial Expression recognition has wide application areas and it needs additional correct and reliable FER system. This work has given a survey on face expression recognition @ IJTSRD | Unique Paper ID – IJTSRD31584 | Volume – 4 | Issue – 4 | May-June 2020 Page 1524