Ufa, Ufa, Russian Federation
An approach to personalized service rendering based on using affective computing technologies is described. The proposed approach consists of considering clients’ emotional states and their individual characteristics in the process of providing services. Rendering services is supplemented by the formalization stages of the client’s emotional state and emotional support. The paper considers online learning as the subject area of research. A general description of the online learning process is given. It is concluded that there is no correction of the learners’ emotional state during the lesson. The dependence of the learners’ knowledge level on their emotional state is revealed. A review of existing approaches to considering learners’ emotional states in the process of online learning is given. Learners’ specific behaviour during the lesson is analysed. The features of academic emotions are also considered. The objective is set to increase the online learning effectiveness by taking into account learners’ emotional states and their individual characteristics and by providing emotional support in the learning process. An approach is proposed to formalise learners’ emotional states based on using facial muscle movements as a universal way of recognizing emotions. The stages of recognizing learners’ emotions during the lesson are also described in detail. The task is set to select emotional support based on the learners’ classification according to their emotional state and their individual characteristics using the nearest neighbour method.
personalized services, affective computing, learners’ emotion recognition, online learning, facial expression recognition, classification, nearest neighbour method
1. Pyriev E.A. Emotions in the Motivation Structure of Educational and Professional Activities of University Students. Vestnik of Kostroma State University. Series: Pedagogy. Psychology. Sociokinetics. 2016;1.
2. Hasnine M.N., Bui H.T.T., Tran T.T.T., Nguyen H.T., Akçapınar G., Ueda H. Students’ Emotion Extraction and Visualization for Engagement Detection in Online Learning, Procedia Computer Science. 2021;192:3423-3431.
3. Liu W., Zhang L., Tao D., Cheng J. Reinforcement Online Learning for Emotion Prediction by Using Physiological Signals. Pattern Recognition Letters. 2018;107:123-130.
4. Cen L, Wu F, Yu ZhL, Hu F. A Real-Time Speech Emotion Recognition System and Its Application in Online Learning. In: Tettegah SY, Gartmeier M, editors. Emotions and Technology, Emotions, Technology, Design, and Learning. Academic Press; 2016. Chapter 2, p. 27-46.
5. Whiteside AL, Dikkers AG. Leveraging the Social Presence Model: A Decade of Research on Emotion in Online and Blended Learning. In: Tettegah ShY, McCreery MP, editors. Emotions and Technology, Emotions, Technology, and Learning. Academic Press; 2016. Chapter 11, p. 225-241.
6. Phirangee Kr, Hewitt J. Loving this Dialogue: Expressing Emotion through the Strategic Manipulation of Limited Non-Verbal Cues in Online Learning Environments. In: Tettegah ShY, McCreery MP, editors. Emotions and Technology, Emotions, Technology, and Learning. Academic Press; 2016. Chapter 4, p. 69-85.
7. Swerdloff M. Online Learning, Multimedia, and Emotions. In: Tettegah ShY, McCreery MP, editors. Emotions and Technology, Emotions, Technology, and Learning. Academic Press; 2016. Chapter 8, p. 155-175.
8. Yang D., Alsadoon A., Prasad P.W.C., Singh A.K., Elchouemi A. An Emotion Recognition Model Based on Facial Recognition in Virtual Learning Environment. Procedia Computer Science. 2018;125:2-10.
9. Imani M., Montazer Gh.A. A Survey of Emotion Recognition Methods with Emphasis on E-Learning Environments. Journal of Network and Computer Applications. 2019;147:102423.
10. Bouhlal M., Aarika K., Abdelouahid R.A., Elfilali S., Benlahmar E. Emotions Recognition as Innovative Tool for Improving Students’ Performance and Learning Approaches. Procedia Computer Science. 2020;175:597-602.
11. Song X, Song Y. Research and Implementation of Online Learning System Based on Electroencephalogram Emotion Computing. In: Proceedings of the 5th International Conference on Mechanical, Control and Computer Engineering (ICMCCE); 2020. p. 1663-1666.
12. Sandanayake TC, Madurapperuma AP. Affective E-Learning Model for Recognising Learner Emotions in Online Learning Environment. In: Proceedings of International Conference on Advances in ICT for Emerging Regions (ICTer); 2013. p. 266-271.
13. Ma C, Sun C, Song D, Li X, Xu H. A Deep Learning Approach for Online Learning Emotion Recognition. In: Proceedings of the 13th International Conference on Computer Science & Education; 2018. p. 1-5.
14. Zhang X, Luo C, He T, Yang X, Lu Z, Huang B. Online Learner Emotional Analysis Based on Big Dataset of Online Learning Forum. In: Proceedings of the 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI); 2017. p. 1-5.
15. Pang JMH, Connie T, Michael GKO. Recognition of Academic Emotions in Online Classes. In: Proceedings of the 9th International Conference on Information and Communication Technology (ICoICT); 2021. p. 445-450.
16. Jain A, Sah HR, Kothari A. Study for Emotion Recognition of Different Age Groups Students during Online Class. In: Proceedings of the 8th International Conference on Computing for Sustainable Global Development (INDIACom); 2021. p. 621-625.
17. Nicolaidou I., Tozzi F., Antoniades A. A Gamified App on Emotion Recognition and Anger Management for Pre-School Children. International Journal of Child-Computer Interaction. 2022;31:100449.
18. Lyu L., Zhang Y., Chi M-Y., Yang F., Zhang Sh-G., Liu P., Lu W.-G. Spontaneous Facial Expression Database of Learners’ Academic Emotions in Online Learning with Hand Occlusion. Computers & Electrical Engineering. 2022;97:107667.
19. Kuruvayil S., Palaniswamy S. Emotion Recognition from Facial Images with Simultaneous Occlusion, Pose and Illumination Variations Using Meta-Learning. Journal of King Saud University - Computer and Information Sciences. 2021.
20. Berweger B., Born S., Dietrich J. Expectancy-Value Appraisals and Achievement Emotions in an Online Learning Environment: Within- and between-Person Relationships, Learning and Instruction. 2022;77:101546.
21. Ekman P., Friesen V.V. Facial Action Coding System: a Technique for the Measuring of Facial Movements. Palo Alto (Cal): Consultations of Psychologists Press; 1978.
22. Yakupova A.V., Smetanina O.N., Sazonova E.Yu. Software Solution of Segmentation Problem Based on Intelligent Technologies. Vestnik of Ufa State Aviation Technical University. 2021;25(3-93):132-144.
23. Iulamanova A., Bogdanova D., Kotelnikov V. Decision Support in the Automated Compilation of Individual Training Module Based on the Emotional State of Students. IFAC-PapersOnLine Series. 2021;54(13):85-90.
