Search
Search results for: HUMAN EMOTION
-
Human emotion recognition with biosignals
PublicationThis chapter presents issues in the field of affective computing. Basic preliminary information for the recognition of emotions is given and models of emotions, various ways of evoking emotions, as well as their theoretical foundations are discussed. The particular attention is given to the use of physiological signals in recognizing emotions. This subject is outlined further below by presenting selected biosignals, their relationship...
-
MACHINE LEARNING APPLICATIONS IN RECOGNIZING HUMAN EMOTIONS BASED ON THE EEG
PublicationThis study examined the machine learning-based approach allowing the recognition of human emotional states with the use of EEG signals. After a short introduction to the fundamentals of electroencephalography and neural oscillations, the two-dimensional valence-arousal Russell’s model of emotion was described. Next, we present the assumptions of the performed EEG experiment. Detail aspects of the data sanitization including preprocessing,...
-
Emotion Monitor - Concept, Construction and Lessons Learned
PublicationThis paper concerns the design and physical construction of an emotion monitor stand for tracking human emotions in Human-Computer Interaction using multi-modal approach. The concept of the stand using cameras, behavioral analysis tools and a set of physiological sensors such as galvanic skin response, blood-volume pulse, temperature, breath and electromyography is presented and followed...
-
Emotion Recognition Using Physiological Signals
PublicationIn this paper the problem of emotion recognition using physiological signals is presented. Firstly the problems with acquisition of physiological signals related to specific human emotions are described. It is not a trivial problem to elicit real emotions and to choose stimuli that always, and for all people, elicit the same emotion. Also different kinds of physiological signals for emotion recognition are considered. A set of...
-
Emotion Monitoring – Verification of Physiological Characteristics Measurement Procedures
PublicationThis paper concerns measurement procedures on an emotion monitoring stand designed for tracking human emotions in the Human-Computer Interaction with physiological characteristics. The paper addresses the key problem of physiological measurements being disturbed by a motion typical for human-computer interaction such as keyboard typing or mouse movements. An original experiment...
-
Identification of Emotions Based on Human Facial Expressions Using a Color-Space Approach
PublicationHCI technology improves human-computer interaction. Such communication can be carried out with the use of emotions that are visible on the human face since birth. In this paper the Emotion system for detecting and recognizing facial expressions, developed in the MSc work, is presented. The system recognizes emotion from webcam video in real time. It is based on color segmentation and morphological operations. The system uses a...
-
Using Physiological Signals for Emotion Recognition
PublicationRecognizing user’s emotions is the promising area of research in a field of human-computer interaction. It is possible to recognize emotions using facial expression, audio signals, body poses, gestures etc. but physiological signals are very useful in this field because they are spontaneous and not controllable. In this paper a problem of using physiological signals for emotion recognition is presented. The kinds of physiological...
-
Towards Emotion Acquisition in IT Usability Evaluation Context
PublicationThe paper concerns extension of IT usability studies with automatic analysis of the emotional state of a user. Affect recognition methods and emotion representation models are reviewed and evaluated for applicability in usability testing procedures. Accuracy of emotion recognition, susceptibility to disturbances, independence on human will and interference with usability testing procedures are...
-
Emotion Recognition - the need for a complete analysis of the phenomenon of expression formation
PublicationThis article shows how complex emotions are. This has been proven by the analysis of the changes that occur on the face. The authors present the problem of image analysis for the purpose of identifying emotions. In addition, they point out the importance of recording the phenomenon of the development of emotions on the human face with the use of high-speed cameras, which allows the detection of micro expression. The work that was...
-
Bimodal Emotion Recognition Based on Vocal and Facial Features
PublicationEmotion recognition is a crucial aspect of human communication, with applications in fields such as psychology, education, and healthcare. Identifying emotions accurately is challenging, as people use a variety of signals to express and perceive emotions. In this study, we address the problem of multimodal emotion recognition using both audio and video signals, to develop a robust and reliable system that can recognize emotions...
-
Analysis of human behavioral patterns
PublicationWidespread usage of Internet and mobile devices entailed growing requirements concerning security which in turn brought about development of biometric methods. However, a specially designed biometric system may infer more about users than just verifying their identity. Proper analysis of users’ characteristics may also tell much about their skills, preferences, feelings. This chapter presents biometric methods applied in several...
-
Modeling emotions for affect-aware applications
PublicationThe chapter concerns emotional states representation and modeling for software systems, that deal with human affect. A review of emotion representation models is provided, including discrete, dimensional and componential models. The paper provides also analysis of emotion models used in diverse types of affect-aware applications: games, mood trackers or tutoring systems. The analysis is supported with two design cases. The study...
-
Investigation of educational processes with affective computing methods
PublicationThis paper concerns the monitoring of educational processes with the use of new technologies for the recognition of human emotions. This paper summarizes results from three experiments, aimed at the validation of applying emotion recognition to e-learning. An analysis of the experiments’ executions provides an evaluation of the emotion elicitation methods used to monitor learners. The comparison of affect recognition algorithms...
-
Robot Eye Perspective in Perceiving Facial Expressions in Interaction with Children with Autism
PublicationThe paper concerns automatic facial expression analysis applied in a study of natural “in the wild” interaction between children with autism and a social robot. The paper reports a study that analyzed the recordings captured via a camera located in the eye of a robot. Children with autism exhibit a diverse level of deficits, including ones in social interaction and emotional expression. The aim of the study was to explore the possibility...
-
Interpretation and modeling of emotions in the management of autonomous robots using a control paradigm based on a scheduling variable
PublicationThe paper presents a technical introduction to psychological theories of emotions. It highlights a usable ideaimplemented in a number of recently developed computational systems of emotions, and the hypothesis thatemotion can play the role of a scheduling variable in controlling autonomous robots. In the main part ofthis study, we outline our own computational system of emotion – xEmotion – designed as a key structuralelement in...
-
Identification of Emotional States Using Phantom Miro M310 Camera
PublicationThe purpose of this paper is to present the possibilities associated with the use of remote sensing methods in identifying human emotional states, and to present the results of the research conducted by the authors in this field. The studies presented involved the use of advanced image analysis to identify areas on the human face that change their activity along with emotional expression. Most of the research carried out in laboratories...
-
Affective Learning Manifesto – 10 Years Later
PublicationIn 2004 a group of affective computing researchers proclaimed a manifesto of affective learning that outlined the prospects and white spots of research at that time. Ten years passed by and affective computing developed many methods and tools for tracking human emotional states as well as models for affective systems construction. There are multiple examples of affective methods applications in Intelligent Tutoring Systems (ITS)....
-
Agnieszka Landowska dr hab. inż.
PeopleAgnieszka Landowska works for Gdansk University of Technology, FETI, Department of Software Engineering. Her research concentrates on usability, accessibility and technology adoption, as well as affective computing methods. She initiated Emotions in HCI Research Group and conducts resarch on User eXperiene evaluation of applications and other technologies.
-
WYKORZYSTANIE SIECI NEURONOWYCH DO SYNTEZY MOWY WYRAŻAJĄCEJ EMOCJE
PublicationW niniejszym artykule przedstawiono analizę rozwiązań do rozpoznawania emocji opartych na mowie i możliwości ich wykorzystania w syntezie mowy z emocjami, wykorzystując do tego celu sieci neuronowe. Przedstawiono aktualne rozwiązania dotyczące rozpoznawania emocji w mowie i metod syntezy mowy za pomocą sieci neuronowych. Obecnie obserwuje się znaczny wzrost zainteresowania i wykorzystania uczenia głębokiego w aplikacjach związanych...
-
IMAGE CORRELATION AS A TOLL FOR TRACKING FACIAL CHANGES CAUSING BY EXTERNAL STIMULI
PublicationExpressions of the human face bring a lot of information, which are a valuable source in the areas of computer vision, remote sensing and affective computing. For years, by analyzing the movement of the skin and facial muscles scientists are trying to create the perfect tool, based on image analysis, allowing the recognition of emotional states of human beings. To create a reliable algorithm, it is necessary to explore and examine...
-
A review of emotion recognition methods based on keystroke dynamics and mouse movements
PublicationThe paper describes the approach based on using standard input devices, such as keyboard and mouse, as sources of data for the recognition of users’ emotional states. A number of systems applying this idea have been presented focusing on three categories of research problems, i.e. collecting and labeling training data, extracting features and training classifiers of emotions. Moreover the advantages and examples of combining standard...
-
AffecTube — Chrome extension for YouTube video affective annotations
PublicationThe shortage of emotion-annotated video datasets suitable for training and validating machine learning models for facial expression-based emotion recognition stems primarily from the significant effort and cost required for manual annotation. In this paper, we present AffecTube as a comprehensive solution that leverages crowdsourcing to annotate videos directly on the YouTube platform, resulting in ready-to-use emotion-annotated...
-
Mining inconsistent emotion recognition results with the multidimensional model
PublicationThe paper deals with the challenge of inconsistency in multichannel emotion recognition. The focus of the paper is to explore factors that might influence the inconsistency. The paper reports an experiment that used multi-camera facial expression analysis with multiple recognition systems. The data were analyzed using a multidimensional approach and data mining techniques. The study allowed us to explore camera location, occlusions...
-
Limitations of Emotion Recognition from Facial Expressions in e-Learning Context
PublicationThe paper concerns technology of automatic emotion recognition applied in e-learning environment. During a study of e-learning process the authors applied facial expressions observation via multiple video cameras. Preliminary analysis of the facial expressions using automatic emotion recognition tools revealed several unexpected results, including unavailability of recognition due to face coverage and significant inconsistency...
-
Emotion Recognition from Physiological Channels Using Graph Neural Network
PublicationIn recent years, a number of new research papers have emerged on the application of neural networks in affective computing. One of the newest trends observed is the utilization of graph neural networks (GNNs) to recognize emotions. The study presented in the paper follows this trend. Within the work, GraphSleepNet (a GNN for classifying the stages of sleep) was adjusted for emotion recognition and validated for this purpose. The...
-
Integration in Multichannel Emotion Recognition
PublicationThe paper concerns integration of results provided by automatic emotion recognition algorithms. It presents both the challenges and the approaches to solve them. Paper shows experimental results of integration. The paper might be of interest to researchers and practitioners who deal with automatic emotion recognition and use more than one solution or multichannel observation.
-
Emotion monitoring system for drivers
PublicationThis article describes a new approach to the issue of building a driver monitoring system. Actual systems focus, for example, on tracking eyelid and eyebrow movements that result from fatigue. We propose a different approach based on monitoring the state of emotions. Such a system assumes that by using the emotion model based on our own concept, referred to as the reverse Plutchik’s paraboloid of emotions, the recognition of emotions...
-
Remote Estimation of Video-Based Vital Signs in Emotion Invocation Studies
PublicationAbstract— The goal of this study is to examine the influence of various imitated and video invoked emotions on the vital signs (respiratory and pulse rates). We also perform an analysis of the possibility to extract signals from sequences acquired with cost-effective cameras. The preliminary results show that the respiratory rate allows for better separation of some emotions than the pulse rate, yet this relation highly depends...
-
The Impact of Lexicon Adaptation on the Emotion Mining From Software Engineering Artifacts
PublicationSentiment analysis and emotion mining techniques are increasingly being used in the field of software engineering. However, the experiments conducted so far have not yielded high accuracy results. Researchers indicate a lack of adaptation of the methods of emotion mining to the specific context of the domain as the main cause of this situation. The article describes research aimed at examining whether the adaptation of the lexicon...
-
Acquisition and indexing of RGB-D recordings for facial expressions and emotion recognition
PublicationIn this paper KinectRecorder comprehensive tool is described which provides for convenient and fast acquisition, indexing and storing of RGB-D video streams from Microsoft Kinect sensor. The application is especially useful as a supporting tool for creation of fully indexed databases of facial expressions and emotions that can be further used for learning and testing of emotion recognition algorithms for affect-aware applications....
-
Emotion recognition and its application in software engineering
PublicationIn this paper a novel application of multimodal emotion recognition algorithms in software engineering is described. Several application scenarios are proposed concerning program usability testing and software process improvement. Also a set of emotional states relevant in that application area is identified. The multimodal emotion recognition method that integrates video and depth channels, physiological signals and input devices...
-
Subjective tests for gathering knowledge for applying color grading to video clips automatically
PublicationThe analysis of film music concerning caused emotions may allow for a more accurate adaptation of the color of the film in the context of color grading. Therefore, this paper aims to gather knowledge on the correlation between the applied color palette to a video clip, music associated with a particular shot, and emotions evoked. For that purpose, subjective tests are prepared in which several video clips are presented with or...
-
Subjective tests for gathering konwledge for applaying color grading to video clips automatically
PublicationThe analysis of film music concerning caused emotions may allow for a more accurate adaptation of the color of the film in the context of color grading. Therefore, this paper aims to gather knowledge on the correlation between the applied color palette to a video clip, music associated with a particular shot,and emotions evoked. For that purpose, subjective tests are prepared in which several video clips are presented with...
-
Towards New Mappings between Emotion Representation Models
PublicationThere are several models for representing emotions in affect-aware applications, and available emotion recognition solutions provide results using diverse emotion models. As multimodal fusion is beneficial in terms of both accuracy and reliability of emotion recognition, one of the challenges is mapping between the models of affect representation. This paper addresses this issue by: proposing a procedure to elaborate new mappings,...
-
Design Elements of Affect Aware Video Games
PublicationIn this paper issues of design and development process of affect-aware video games are presented. Several important design aspects of such games are pointed out. A concept of a middleware framework is proposed that separates the development of affect-aware video games from emotion recognition algorithms and support from input sensors. Finally, two prototype affect-aware video games are presented that conform to the presented architecture...
-
Predicting emotion from color present in images and video excerpts by machine learning
PublicationThis work aims at predicting emotion based on the colors present in images and video excerpts using a machine-learning approach. The purpose of this paper is threefold: (a) to develop a machine-learning algorithm that classifies emotions based on the color present in an image, (b) to select the best-performing algorithm from the first phase and apply it to film excerpt emotion analysis based on colors, (c) to design an online survey...
-
Uncertainty in emotion recognition
PublicationPurpose–The purpose of this paper is to explore uncertainty inherent in emotion recognition technologiesand the consequences resulting from that phenomenon.Design/methodology/approach–The paper is a general overview of the concept; however, it is basedon a meta-analysis of multiple experimental and observational studies performed over the past couple of years.Findings–The mainfinding of the paper might be summarized as follows:...
-
A Study of Cross-Linguistic Speech Emotion Recognition Based on 2D Feature Spaces
PublicationIn this research, a study of cross-linguistic speech emotion recognition is performed. For this purpose, emotional data of different languages (English, Lithuanian, German, Spanish, Serbian, and Polish) are collected, resulting in a cross-linguistic speech emotion dataset with the size of more than 10.000 emotional utterances. Despite the bi-modal character of the databases gathered, our focus is on the acoustic representation...
-
DevEmo—Software Developers’ Facial Expression Dataset
PublicationThe COVID-19 pandemic has increased the relevance of remote activities and digital tools for education, work, and other aspects of daily life. This reality has highlighted the need for emotion recognition technology to better understand the emotions of computer users and provide support in remote environments. Emotion recognition can play a critical role in improving the remote experience and ensuring that individuals are able...
-
Affect aware video games
PublicationIn this chapter a problem of affect aware video games is described, including such issue as: emotional model of the player, design, development and UX testing of affect-aware video games, multimodal emotion recognition and a featured review of affect-aware video games.
-
Introduction to the special issue on machine learning in acoustics
PublicationWhen we started our Call for Papers for a Special Issue on “Machine Learning in Acoustics” in the Journal of the Acoustical Society of America, our ambition was to invite papers in which machine learning was applied to all acoustics areas. They were listed, but not limited to, as follows: • Music and synthesis analysis • Music sentiment analysis • Music perception • Intelligent music recognition • Musical source separation • Singing...
-
Ranking Speech Features for Their Usage in Singing Emotion Classification
PublicationThis paper aims to retrieve speech descriptors that may be useful for the classification of emotions in singing. For this purpose, Mel Frequency Cepstral Coefficients (MFCC) and selected Low-Level MPEG 7 descriptors were calculated based on the RAVDESS dataset. The database contains recordings of emotional speech and singing of professional actors presenting six different emotions. Employing the algorithm of Feature Selection based...
-
Using Different Information Channels for Affect-Aware Video Games - A Case Study
PublicationThis paper presents the problem of creating affect-aware video games that use different information channels, such as image, video, physiological signals, input devices, and player’s behaviour, for emotion recognition. Presented case studies of three affect-aware games show certain conditions and limitations for using specific signals to recognize emotions and lead to interesting conclusions.
-
Emotional distress, burnout and sense of safety during the COVID-19 pandemic in teachers after the reopening of schools
PublicationThe COVID-19 pandemic is having a significant impact on people's psychological well-being and mental health. This study aimed to identify factors linked to emotional distress, burnout and sense of safety in teachers related to the reopening of Polish schools after lockdown, remote work, and the holiday period between March and August 2020. A total of 1,286 teachers from different educational institutions participated in the...
-
A Review of Emotion Recognition Methods Based on Data Acquired via Smartphone Sensors
PublicationIn recent years, emotion recognition algorithms have achieved high efficiency, allowing the development of various affective and affect-aware applications. This advancement has taken place mainly in the environment of personal computers offering the appropriate hardware and sufficient power to process complex data from video, audio, and other channels. However, the increase in computing and communication capabilities of smartphones,...
-
Affective reactions to playing digital games
PublicationThe paper presents a study of emotional states during a gameplay. An experiment of two-player Tetris game is reported, followed by the analysis of the results - self-reported emotional states as well as physiological signals measurements interpretation. The study reveals the diversity of emotional reactions and concludes, that a representative player's emotional model is hard to define. Instead, an adaptive approach to emotion...
-
Emotion Recognition and Its Applications
PublicationThe paper proposes a set of research scenarios to be applied in four domains: software engineering, website customization, education and gaming. The goal of applying the scenarios is to assess the possibility of using emotion recognition methods in these areas. It also points out the problems of defining sets of emotions to be recognized in different applications, representing the defined emotional states, gathering the data and...
-
Recognition of Emotions in Speech Using Convolutional Neural Networks on Different Datasets
PublicationArtificial Neural Network (ANN) models, specifically Convolutional Neural Networks (CNN), were applied to extract emotions based on spectrograms and mel-spectrograms. This study uses spectrograms and mel-spectrograms to investigate which feature extraction method better represents emotions and how big the differences in efficiency are in this context. The conducted studies demonstrated that mel-spectrograms are a better-suited...
-
Comparison of selected off-the-shelf solutions for emotion recognition based on facial expressions
PublicationThe paper concerns accuracy of emotion recognition from facial expressions. As there are a couple of ready off-the-shelf solutions available in the market today, this study aims at practical evaluation of selected solutions in order to provide some insight into what potential buyers might expect. Two solutions were compared: FaceReader by Noldus and Xpress Engine by QuantumLab. The performed evaluation revealed that the recognition...
-
The role of time perspectives and impulsivity dimensions in coping styles
PublicationBoth time perspectives and impulsivity dimensions are groups of traits that are connected to self-control abilities and might be important for coping styles. However, to date, no study has systematically investigated their utility in predicting coping styles with regard to their multidimensional nature. The current study was correlational and exploratory, aiming to discover what amount of variance in each of the three coping...