This study investigates individuals’ cognitive load processing abilities while engaged on a decision-making task in serious games, to explore how a substantial cognitive load dominates over the physiological arousal effect on pupil diameter. A serious game was presented to the participants, which displayed the on–line biofeedback based on physiological measurements of arousal. In such dynamic decision-making environment, the pupil diameter was analyzed in relation to the heart rate, to evaluate if the former could be a useful measure of cognitive abilities of individuals. As pupil might reflect both cognitive activity and physiological arousal, the pupillary response will show an arousal effect only when the cognitive demands of the situation are minimal. Evidence shows that in a situation where a substantial level of cognitive activity is required, only that activity will be observable on the pupil diameter, dominating over the physiological arousal effect indicated by the pupillary response. It is suggested that it might be possible to design serious games tailored to the cognitive abilities of an individual player, using the proposed physiological measurements to observe the moment when such dominance occurs. © 2018, The Author(s).

Elicited physiological affect in humans collaborating with their robot partners was investigated to determine its influence on decision-making performance in serious games. A turn-taking version of the Tower of Hanoi game was used, where physiological arousal and valence underlying such human-robot proximate collaboration were investigated. A comparable decision performance in the serious game was found between human and non-humanoid robot arm collaborator conditions, while higher physiological affect was found in humans collaborating with such robot collaborators. It is suggested that serious games which are carefully designed to take into consideration the elicited physiological arousal might witness a better decision-making performance and more positive valence using non-humanoid robot partners instead of human ones. © 2019 The Authors

This paper sets out to investigate the potentials of using pupil diameter measure as a contactless biofeedback method. The investigation was performed on how the interdependent and competing activation of the autonomic nervous system is reflected in the pupil diameter and how it affects the performance on decision-making task in serious games. The on-line biofeedback based on physiological measurements of arousal was integrated into the serious game set in the financial context. The pupil diameter was validated against the heart rate data measuring arousal, where the effects of such arousal were investigated. It was found that the physiological arousal was observable on both the heart and pupil data. Furthermore, the participants with lower arousal took less time to reach their decisions, and those decisions were more successful, in comparison to the participants with higher arousal. Moreover, such participants were able to get a higher total score and finish the game. This study validated the potential usage of pupil diameter as an unobtrusive measure of biofeedback, which would be beneficial for the investigation of arousal on human decision-making inside of serious games. © IFIP International Federation for Information Processing, 2019.

Evidence shows that emotions critically influence human decision-making. Therefore, emotion-regulation using biofeedback has been extensively investigated. Nevertheless, serious games have emerged as a valuable tool for such investigations set in the decision-making context. This review sets out to investigate the scientific evidence regarding the effects of practicing emotion-regulation through biofeedback on the decision-making performance in the context of serious games. A systematic search of five electronic databases (Scopus, Web of Science, IEEE, PubMed Central, Science Direct), followed by the author and snowballing investigation, was conducted from a publication's year of inception to October 2018. The search identified 16 randomized controlled experiment/quasi-experiment studies that quantitatively assessed the performance on decision-making tasks in serious games, involving students, military, and brain-injured participants. It was found that the participants who raised awareness of emotions and increased the skill of emotion-regulation were able to successfully regulate their arousal, which resulted in better decision performance, reaction time, and attention scores on the decision-making tasks. It is suggested that serious games provide an effective platform validated through the evaluative and playtesting studies, that supports the acquisition of the emotion-regulation skill through the direct (visual) and indirect (gameplay) biofeedback presentation on decision-making tasks.

Emotions are thought to be one of the key factors that critically influence human decision-making. Emotion-regulation can help to mitigate emotion-related decision biases and eventually lead to a better decision performance. Serious games emerged as a new angle introducing technological methods to practicing emotion-regulation, where meaningful biofeedback information communicates player's affective states to a series of informed gameplay choices. These findings motivate the notion that in the decision context of serious games, one would benefit from awareness and regulation of such emerging emotions.

This thesis explores the design and evaluation methods for creating serious games where emotion-regulation can be practiced using physiological biofeedback measures. Furthermore, it investigates emotions and the effect of emotion-regulation on decision performance in serious games. Using the psychophysiological methods in the design of such games, emotions and their underlying neural mechanism have been explored.

The results showed the benefits of practicing emotion-regulation in serious games, where decision-making performance was increased for the individuals who down-regulated high levels of arousal while having an experience of positive valence. Moreover, it increased also for the individuals who received the necessary biofeedback information. The results also suggested that emotion-regulation strategies (i.e., cognitive reappraisal) are highly dependent on the serious game context. Therefore, the reappraisal strategy was shown to benefit the decision-making tasks investigated in this thesis. The results further suggested that using psychophysiological methods in emotionally arousing serious games, the interplay between sympathetic and parasympathetic pathways could be mapped through the underlying emotions which activate those two pathways. Following this conjecture, the results identified the optimal arousal level for increased performance of an individual on a decision-making task, by carefully balancing the activation of those two pathways. The investigations also validated these findings in the collaborative serious game context, where the robot collaborators were found to elicit diverse affect in their human partners, influencing performance on a decision-making task. Furthermore, the evidence suggested that arousal is equally or more important than valence for the decision-making performance, but once optimal arousal has been reached, a further increase in performance may be achieved by regulating valence. Furthermore, the results showed that serious games designed in this thesis elicited high physiological arousal and positive valence. This makes them suitable as research platforms for the investigation of how these emotions influence the activation of sympathetic and parasympathetic pathways and influence performance on a decision-making task.

Taking these findings into consideration, the serious games designed in this thesis allowed for the training of cognitive reappraisal emotion-regulation strategy on the decision-making tasks. This thesis suggests that using evaluated design and development methods, it is possible to design and develop serious games that provide a helpful environment where individuals could practice emotion-regulation through raising awareness of emotions, and subsequently improve their decision-making performance.

This paper sets out to investigate how the basal activation of the parasympathetic and sympathetic nervous systems may affect and predict the decision-making performance of players in serious games. In order to investigate the basal activation of both branches of the autonomic nervous system, pupil diameter and heart rate were recorded during baseline and analyzed in regards to performance scores in the serious game. It was found that the balance between the parasympathetic and sympathetic activation was responsible for beneficial decision-making performance, while lower sympathetic activation was found to be associated with the higher level reached in the game. It is suggested that the balance between the basal activation of both branches of the autonomic nervous system recorded during the baseline may predict the decision-making performance of players on the subsequent tasks in serious games. © 2019 IEEE.

We report on a pilot study involving emotion elicitation in virtual reality (VR) and assessment of emotional responses with a consumer-grade EEG device. The stimulation used HTC Vive VR system showing pictures from NAPS database within a specifically designed virtual environment. The stimulation consisted of two distinct sequences with 10 pictures of happiness and 10 pictures of fear. Each picture was contained in a separate virtual room that the participants traveled through along a preset path. The estimation employed EMOTIV EPOC+ 14-channel EEG headset and a custom-developed application. The software wirelessly received EEG signals from alpha, beta low, beta high, gamma and theta bands, time-stamped them and dynamically stored in a relational database for subsequent analysis. Our preliminary results show that statistically significant correlations between valence and arousal ratings of pictures and EEG bands are present but highly personalized. Simultaneous correct placement of VR and EEG headsets is demanding and precise localization of electrodes is difficult. In fact, if emotion estimation is not strictly necessary we recommend using devices with fewer electrodes. Nevertheless, we found the EEG to be effective. By acknowledging its limitations, and using the headset in the correct context, experiments involving emotions may be significantly amended. © 2018 Croatian Society MIPRO.

This paper sets out to examine how elicited physiological affect influences the performance of human participants collaborating with the robot partners on a shared serious game task; furthermore, to investigate physiological affect underlying such human-robot proximate collaboration. The participants collaboratively played a turn-taking version of a serious game Tower of Hanoi, where physiological affect was investigated in a valence-arousal space. The arousal was inferred from the galvanic skin response data, while the valence was inferred from the electrocardiography data. It was found that the robot collaborators elicited a higher physiological affect in regard to both arousal and valence, in contrast to their human collaborator counterparts. Furthermore, a comparable performance between all collaborators was found on the serious game task. © 2018, IFIP International Federation for Information Processing.

The aim of this paper is to investigate performance in a collaborative human–robot interaction on a shared serious game task. Furthermore, the effect of elicited emotions and perceived social behavior categories on players’ performance will be investigated. The participants collaboratively played a turn-taking version of the Tower of Hanoi serious game, together with the human and robot collaborators. The elicited emotions were analyzed in regards to the arousal and valence variables, computed from the Geneva Emotion Wheel questionnaire. Moreover, the perceived social behavior categories were obtained from analyzing and grouping replies to the Interactive Experiences and Trust and Respect questionnaires. It was found that the results did not show a statistically significant difference in participants’ performance between the human or robot collaborators. Moreover, all of the collaborators elicited similar emotions, where the human collaborator was perceived as more credible and socially present than the robot one. It is suggested that using robot collaborators might be as efficient as using human ones, in the context of serious game collaborative tasks.

The aim of this paper is to investigate how a substantial cognitive load overshadows the physiological arousal effect, in an attempt to study cognitive abilities of participants engaged on decision-making tasks in serious games. Participants were engaged in a dynamic serious game environment displaying online biofeedback based on the physiological measurements of arousal. The pupil diameter was analyzed in relation to the heart rate during a challenging decision-making task. It was found that the moment when a substantial cognitive load overshadows the physiological arousal effect is observable on the pupil diameter in relation to the heart rate. © 2017 IEEE.