Change search
Link to record
Permanent link

Direct link
BETA
Jerčić, Petar
Alternative names
Publications (10 of 14) Show all publications
Jerčić, P. & Sundstedt, V. (2019). Practicing Emotion-Regulation Through Biofeedback on the Decision-Making Performance in the Context of Serious Games: a Systematic Review. Entertainment Computing, 29, 75-86
Open this publication in new window or tab >>Practicing Emotion-Regulation Through Biofeedback on the Decision-Making Performance in the Context of Serious Games: a Systematic Review
2019 (English)In: Entertainment Computing, ISSN 1875-9521, E-ISSN 1875-953X, Vol. 29, p. 75-86Article in journal (Refereed) Published
Abstract [en]

Evidence shows that emotions critically influence human decision-making. Therefore, emotion-regulation using biofeedback has been extensively investigated. Nevertheless, serious games have emerged as a valuable tool for such investigations set in the decision-making context. This review sets out to investigate the scientific evidence regarding the effects of practicing emotion-regulation through biofeedback on the decision-making performance in the context of serious games. A systematic search of five electronic databases (Scopus, Web of Science, IEEE, PubMed Central, Science Direct), followed by the author and snowballing investigation, was conducted from a publication's year of inception to October 2018. The search identified 16 randomized controlled experiment/quasi-experiment studies that quantitatively assessed the performance on decision-making tasks in serious games, involving students, military, and brain-injured participants. It was found that the participants who raised awareness of emotions and increased the skill of emotion-regulation were able to successfully regulate their arousal, which resulted in better decision performance, reaction time, and attention scores on the decision-making tasks. It is suggested that serious games provide an effective platform validated through the evaluative and playtesting studies, that supports the acquisition of the emotion-regulation skill through the direct (visual) and indirect (gameplay) biofeedback presentation on decision-making tasks.

Place, publisher, year, edition, pages
Elsevier, 2019
Keywords
serious games, emotions, affect, biofeedback, emotion-regulation, decision-making
National Category
Computer Systems
Identifiers
urn:nbn:se:bth-17580 (URN)10.1016/j.entcom.2019.01.001 (DOI)000458213700008 ()
Note

open access

Available from: 2019-02-05 Created: 2019-02-05 Last updated: 2019-03-04Bibliographically approved
Jerčić, P. (2019). The Effects of Emotions and Their Regulation on Decision-making Performance in Affective Serious Games. (Doctoral dissertation). Karlskrona: Blekinge Tekniska Högskola
Open this publication in new window or tab >>The Effects of Emotions and Their Regulation on Decision-making Performance in Affective Serious Games
2019 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Emotions are thought to be one of the key factors that critically influence human decision-making. Emotion-regulation can help to mitigate emotion-related decision biases and eventually lead to a better decision performance. Serious games emerged as a new angle introducing technological methods to practicing emotion-regulation, where meaningful biofeedback information communicates player's affective states to a series of informed gameplay choices. These findings motivate the notion that in the decision context of serious games, one would benefit from awareness and regulation of such emerging emotions.

This thesis explores the design and evaluation methods for creating serious games where emotion-regulation can be practiced using physiological biofeedback measures. Furthermore, it investigates emotions and the effect of emotion-regulation on decision performance in serious games. Using the psychophysiological methods in the design of such games, emotions and their underlying neural mechanism have been explored.

The results showed the benefits of practicing emotion-regulation in serious games, where decision-making performance was increased for the individuals who down-regulated high levels of arousal while having an experience of positive valence. Moreover, it increased also for the individuals who received the necessary biofeedback information. The results also suggested that emotion-regulation strategies (i.e., cognitive reappraisal) are highly dependent on the serious game context. Therefore, the reappraisal strategy was shown to benefit the decision-making tasks investigated in this thesis. The results further suggested that using psychophysiological methods in emotionally arousing serious games, the interplay between sympathetic and parasympathetic pathways could be mapped through the underlying emotions which activate those two pathways. Following this conjecture, the results identified the optimal arousal level for increased performance of an individual on a decision-making task, by carefully balancing the activation of those two pathways. The investigations also validated these findings in the collaborative serious game context, where the robot collaborators were found to elicit diverse affect in their human partners, influencing performance on a decision-making task. Furthermore, the evidence suggested that arousal is equally or more important than valence for the decision-making performance, but once optimal arousal has been reached, a further increase in performance may be achieved by regulating valence. Furthermore, the results showed that serious games designed in this thesis elicited high physiological arousal and positive valence. This makes them suitable as research platforms for the investigation of how these emotions influence the activation of sympathetic and parasympathetic pathways and influence performance on a decision-making task.

Taking these findings into consideration, the serious games designed in this thesis allowed for the training of cognitive reappraisal emotion-regulation strategy on the decision-making tasks. This thesis suggests that using evaluated design and development methods, it is possible to design and develop serious games that provide a helpful environment where individuals could practice emotion-regulation through raising awareness of emotions, and subsequently improve their decision-making performance.

Place, publisher, year, edition, pages
Karlskrona: Blekinge Tekniska Högskola, 2019. p. 297
Series
Blekinge Institute of Technology Doctoral Dissertation Series, ISSN 1653-2090 ; 6
Keywords
Serious Games, Game Design, Emotions, Biofeedback, Emotion-Regulation, Decision-Making
National Category
Computer Systems
Identifiers
urn:nbn:se:bth-17557 (URN)978-91-7295-370-3 (ISBN)
Public defence
2019-04-26, J1640, Campus Gräsvik, Karlskrona, 09:12 (English)
Opponent
Supervisors
Available from: 2019-01-31 Created: 2019-01-30 Last updated: 2019-05-09Bibliographically approved
Horvat, M., Dobrinic, M., Novosel, M. & Jerčić, P. (2018). Assessing emotional responses induced in virtual reality using a consumer EEG headset: A preliminary report. In: 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics, MIPRO 2018 - Proceedings: . Paper presented at 41st International Convention on Information and Communication Technology, Electronics and Microelectronics, MIPRO , Opatija, Croatia (pp. 1006-1010). Institute of Electrical and Electronics Engineers Inc.
Open this publication in new window or tab >>Assessing emotional responses induced in virtual reality using a consumer EEG headset: A preliminary report
2018 (English)In: 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics, MIPRO 2018 - Proceedings, Institute of Electrical and Electronics Engineers Inc. , 2018, p. 1006-1010Conference paper, Published paper (Refereed)
Abstract [en]

We report on a pilot study involving emotion elicitation in virtual reality (VR) and assessment of emotional responses with a consumer-grade EEG device. The stimulation used HTC Vive VR system showing pictures from NAPS database within a specifically designed virtual environment. The stimulation consisted of two distinct sequences with 10 pictures of happiness and 10 pictures of fear. Each picture was contained in a separate virtual room that the participants traveled through along a preset path. The estimation employed EMOTIV EPOC+ 14-channel EEG headset and a custom-developed application. The software wirelessly received EEG signals from alpha, beta low, beta high, gamma and theta bands, time-stamped them and dynamically stored in a relational database for subsequent analysis. Our preliminary results show that statistically significant correlations between valence and arousal ratings of pictures and EEG bands are present but highly personalized. Simultaneous correct placement of VR and EEG headsets is demanding and precise localization of electrodes is difficult. In fact, if emotion estimation is not strictly necessary we recommend using devices with fewer electrodes. Nevertheless, we found the EEG to be effective. By acknowledging its limitations, and using the headset in the correct context, experiments involving emotions may be significantly amended. © 2018 Croatian Society MIPRO.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers Inc., 2018
Keywords
Electrodes, Microelectronics, Developed applications, Emotion elicitation, Emotion estimation, Emotional response, Emotiv epoc, Pilot studies, Relational Database, Virtual rooms, Virtual reality
National Category
Other Electrical Engineering, Electronic Engineering, Information Engineering
Identifiers
urn:nbn:se:bth-16908 (URN)10.23919/MIPRO.2018.8400184 (DOI)2-s2.0-85050249600 (Scopus ID)9789532330977 (ISBN)
Conference
41st International Convention on Information and Communication Technology, Electronics and Microelectronics, MIPRO , Opatija, Croatia
Available from: 2018-08-20 Created: 2018-08-20 Last updated: 2018-08-20Bibliographically approved
Jerčić, P., Sennersten, C. & Lindley, C. (2018). Modeling cognitive load and physiological arousal through pupil diameter and heart rate. Multimedia tools and applications
Open this publication in new window or tab >>Modeling cognitive load and physiological arousal through pupil diameter and heart rate
2018 (English)In: Multimedia tools and applications, ISSN 1380-7501, E-ISSN 1573-7721Article in journal (Refereed) Epub ahead of print
Abstract [en]

This study investigates individuals’ cognitive load processing abilities while engaged on a decision-making task in serious games, to explore how a substantial cognitive load dominates over the physiological arousal effect on pupil diameter. A serious game was presented to the participants, which displayed the on–line biofeedback based on physiological measurements of arousal. In such dynamic decision-making environment, the pupil diameter was analyzed in relation to the heart rate, to evaluate if the former could be a useful measure of cognitive abilities of individuals. As pupil might reflect both cognitive activity and physiological arousal, the pupillary response will show an arousal effect only when the cognitive demands of the situation are minimal. Evidence shows that in a situation where a substantial level of cognitive activity is required, only that activity will be observable on the pupil diameter, dominating over the physiological arousal effect indicated by the pupillary response. It is suggested that it might be possible to design serious games tailored to the cognitive abilities of an individual player, using the proposed physiological measurements to observe the moment when such dominance occurs. © 2018, The Author(s).

Place, publisher, year, edition, pages
Springer New York LLC, 2018
Keywords
Arousal, Cognitive load, Electrocardiogram, Physiology, Pupil diameter, Serious games, Biofeedback, Decision making, Electrocardiography, Heart, Neurophysiology, Physiological models, Cognitive activities, Cognitive loads, Dynamic decision making, Physiological measurement, Processing ability, Pupillary response, Psychophysiology
National Category
Other Computer and Information Science
Identifiers
urn:nbn:se:bth-17052 (URN)10.1007/s11042-018-6518-z (DOI)2-s2.0-85053407843 (Scopus ID)
Note

open access

Available from: 2018-09-27 Created: 2018-09-27 Last updated: 2019-01-30Bibliographically approved
Jerčić, P., Hagelbäck, J. & Lindley, C. (2018). Physiological Affect and Performance in a Collaborative Serious Game Between Humans and an Autonomous Robot. In: Lect. Notes Comput. Sci.: . Paper presented at 17th IFIP TC 14 International Conference on Entertainment Computing, ICEC, Poznan (pp. 127-138). Springer Verlag, 11112
Open this publication in new window or tab >>Physiological Affect and Performance in a Collaborative Serious Game Between Humans and an Autonomous Robot
2018 (English)In: Lect. Notes Comput. Sci., Springer Verlag , 2018, Vol. 11112, p. 127-138Conference paper, Published paper (Refereed)
Abstract [en]

This paper sets out to examine how elicited physiological affect influences the performance of human participants collaborating with the robot partners on a shared serious game task; furthermore, to investigate physiological affect underlying such human-robot proximate collaboration. The participants collaboratively played a turn-taking version of a serious game Tower of Hanoi, where physiological affect was investigated in a valence-arousal space. The arousal was inferred from the galvanic skin response data, while the valence was inferred from the electrocardiography data. It was found that the robot collaborators elicited a higher physiological affect in regard to both arousal and valence, in contrast to their human collaborator counterparts. Furthermore, a comparable performance between all collaborators was found on the serious game task. © 2018, IFIP International Federation for Information Processing.

Place, publisher, year, edition, pages
Springer Verlag, 2018
Series
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), ISSN 03029743
Keywords
Affect, Autonomous robots, Collaborative play, Emotions, Physiology, Robot-assisted play, Serious games, Electrophysiology, Robots, Galvanic skin response, Human robots, Robot Partners, Tower of Hanoi, Turn-taking
National Category
Computer Sciences
Identifiers
urn:nbn:se:bth-17087 (URN)10.1007/978-3-319-99426-0_11 (DOI)2-s2.0-85053772889 (Scopus ID)9783319994253 (ISBN)
Conference
17th IFIP TC 14 International Conference on Entertainment Computing, ICEC, Poznan
Available from: 2018-10-05 Created: 2018-10-05 Last updated: 2019-01-30Bibliographically approved
Jerčić, P., Wen, W., Hagelbäck, J. & Sundstedt, V. (2018). The Effect of Emotions and Social Behavior on Performance in a Collaborative Serious Game Between Humans and Autonomous Robots. International Journal of Social Robotics, 10(1), 115-129
Open this publication in new window or tab >>The Effect of Emotions and Social Behavior on Performance in a Collaborative Serious Game Between Humans and Autonomous Robots
2018 (English)In: International Journal of Social Robotics, ISSN 1875-4791, E-ISSN 1875-4805, Vol. 10, no 1, p. 115-129Article in journal (Refereed) Published
Abstract [en]

The aim of this paper is to investigate performance in a collaborative human–robot interaction on a shared serious game task. Furthermore, the effect of elicited emotions and perceived social behavior categories on players’ performance will be investigated. The participants collaboratively played a turn-taking version of the Tower of Hanoi serious game, together with the human and robot collaborators. The elicited emotions were analyzed in regards to the arousal and valence variables, computed from the Geneva Emotion Wheel questionnaire. Moreover, the perceived social behavior categories were obtained from analyzing and grouping replies to the Interactive Experiences and Trust and Respect questionnaires. It was found that the results did not show a statistically significant difference in participants’ performance between the human or robot collaborators. Moreover, all of the collaborators elicited similar emotions, where the human collaborator was perceived as more credible and socially present than the robot one. It is suggested that using robot collaborators might be as efficient as using human ones, in the context of serious game collaborative tasks.

Place, publisher, year, edition, pages
Springer, 2018
Keywords
Autonomous robots, Serious games, Collaborative play, Social interaction, Robot-assisted play, Emotions
National Category
Computer Systems
Identifiers
urn:nbn:se:bth-15541 (URN)10.1007/s12369-017-0437-4 (DOI)000423152900008 ()
Projects
PsyIntEC
Funder
EU, FP7, Seventh Framework Programme, FP7-ICT-231143
Note

open access

Available from: 2017-11-28 Created: 2017-11-28 Last updated: 2019-01-30Bibliographically approved
Jerčić, P., Sennersten, C. & Lindley, C. (2017). The effect of cognitive load on physiological arousal in a decision-making serious game. In: 2017 9th International Conference on Virtual Worlds and Games for Serious Applications, VS-Games 2017 - Proceedings: . Paper presented at 9th International Conference on Virtual Worlds and Games for Serious Applications, VS-Games, Athens (pp. 153-156). Institute of Electrical and Electronics Engineers Inc., Article ID 8056587.
Open this publication in new window or tab >>The effect of cognitive load on physiological arousal in a decision-making serious game
2017 (English)In: 2017 9th International Conference on Virtual Worlds and Games for Serious Applications, VS-Games 2017 - Proceedings, Institute of Electrical and Electronics Engineers Inc. , 2017, p. 153-156, article id 8056587Conference paper, Published paper (Refereed)
Abstract [en]

The aim of this paper is to investigate how a substantial cognitive load overshadows the physiological arousal effect, in an attempt to study cognitive abilities of participants engaged on decision-making tasks in serious games. Participants were engaged in a dynamic serious game environment displaying online biofeedback based on the physiological measurements of arousal. The pupil diameter was analyzed in relation to the heart rate during a challenging decision-making task. It was found that the moment when a substantial cognitive load overshadows the physiological arousal effect is observable on the pupil diameter in relation to the heart rate. © 2017 IEEE.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers Inc., 2017
Series
International Conference on Games and Virtual Worlds for Serious Applications, ISSN 2474-0470
Keywords
arousal, cognitive load, electrocardiogram, physiology, pupil diameter, serious games, Biofeedback, Decision making, Electrocardiography, Heart, Neurophysiology, Psychophysiology, Virtual reality, Cognitive ability, Cognitive loads, Game environment, Heart rates, Physiological measurement
National Category
Human Computer Interaction
Identifiers
urn:nbn:se:bth-15606 (URN)10.1109/VS-GAMES.2017.8056587 (DOI)000425228700023 ()2-s2.0-85034631890 (Scopus ID)9781509058129 (ISBN)
Conference
9th International Conference on Virtual Worlds and Games for Serious Applications, VS-Games, Athens
Available from: 2017-12-14 Created: 2017-12-14 Last updated: 2018-10-19Bibliographically approved
Hagelbäck, J., Hilborn, O., Jercic, P., Johansson, S., Lindley, C., Svensson, J. & Wen, W. (2014). Psychophysiological Interaction and Empathic Cognition for Human-Robot Cooperative Work (PsyIntEC). In: Rohrbein, F.; Veiga, G.; Natale, C. (Ed.), Gearing Up and Accelerating Cross-Fertilization between Academic and Industrial Robotics Research in Europe: Technology Transfer Experiments from the ECHORD Project (pp. 283-299). Springer
Open this publication in new window or tab >>Psychophysiological Interaction and Empathic Cognition for Human-Robot Cooperative Work (PsyIntEC)
Show others...
2014 (English)In: Gearing Up and Accelerating Cross-Fertilization between Academic and Industrial Robotics Research in Europe: Technology Transfer Experiments from the ECHORD Project / [ed] Rohrbein, F.; Veiga, G.; Natale, C., Springer , 2014, p. 283-299Chapter in book (Refereed)
Abstract [en]

The aim of the PsyIntEC project is to explore affective and cognitive modeling of humans in human-robot interaction (HRI) as a basis for behavioral adaptation. To achieve this we have explored human affective perception of relevant modalities in human-human and human-robot interaction on a collaborative problem-solving task using psychophysiological measurements. The experiments conducted have given us valuable insight into the communicational and affective queues interplaying in such interactions from the human perspective. The results indicate that there is an increase in both positive and negative emotions when interacting with robots compared to interacting with another human or solving the task alone, but detailed analysis on shorter time segments is required for the results from all sensors to be conclusive and significant.

Place, publisher, year, edition, pages
Springer, 2014
Keywords
human-robot interaction, psychophysiology, affective modeling, robotics
National Category
Human Aspects of ICT Computer Sciences
Identifiers
urn:nbn:se:bth-6590 (URN)10.1007/978-3-319-02934-4_14 (DOI)000329752400016 ()oai:bth.se:forskinfoB5643A721B059E9AC1257D6D0036DE33 (Local ID)978-3-319-02933-7 (ISBN)oai:bth.se:forskinfoB5643A721B059E9AC1257D6D0036DE33 (Archive number)oai:bth.se:forskinfoB5643A721B059E9AC1257D6D0036DE33 (OAI)
Available from: 2014-10-10 Created: 2014-10-10 Last updated: 2018-01-11Bibliographically approved
Jerčić, P. (2013). Design and Evaluation of Affective Serious Games for Emotion Regulation Training. (Licentiate dissertation). Karlskrona: Blekinge Institute of Technology
Open this publication in new window or tab >>Design and Evaluation of Affective Serious Games for Emotion Regulation Training
2013 (English)Licentiate thesis, comprehensive summary (Other academic)
Abstract [en]

Emotions are thought to be a key factor that critically influences human decision-making. Emotion regulation can help to mitigate emotion related decision biases and eventually lead to a better decision performance. Serious games emerged as a new angle introducing technological methods to learning emotion regulation, where meaningful biofeedback information displays player's emotional state. This thesis investigates emotions and the effect of emotion regulation on decision performance. Furthermore, it explores design and evaluation methods for creating serious games where emotion regulation can be learned and practiced. The scope of this thesis was limited to serious games for emotion regulation training using psychophysiological methods to communicate user's affective information. Using the psychophysiological methods, emotions and their underlying neural mechanism have been explored. Through design and evaluation of serious games using those methods, effects of emotion regulation have been investigated where decision performance has been measured and analyzed. The proposed metrics for designing and evaluating such affective serious games have been exhaustively evaluated. The research methods used in this thesis were based on both quantitative and qualitative aspects, with true experiment and evaluation research, respectively. Serious games approach to emotion regulation was investigated. The results suggested that two different emotion regulation strategies, suppression and cognitive reappraisal, are optimal for different decision tasks contexts. With careful design methods, valid serious games for training those different strategies could be produced. Moreover, using psychophysiological methods, underlying emotion neural mechanism could be mapped to provide optimal level of arousal for a certain task. The results suggest that it is possible to design and develop serious game applications that provide helpful learning environment where decision makers could practice emotion regulation and subsequently improve their decision making.

Place, publisher, year, edition, pages
Karlskrona: Blekinge Institute of Technology, 2013
Series
Blekinge Institute of Technology Licentiate Dissertation Series, ISSN 1650-2140 ; 11
National Category
Computer Sciences Human Aspects of ICT Psychology
Identifiers
urn:nbn:se:bth-00567 (URN)oai:bth.se:forskinfoAC4B5D7201E92CF0C1257BF80045FA91 (Local ID)978-91-7295-266-9 (ISBN)oai:bth.se:forskinfoAC4B5D7201E92CF0C1257BF80045FA91 (Archive number)oai:bth.se:forskinfoAC4B5D7201E92CF0C1257BF80045FA91 (OAI)
Available from: 2014-01-08 Created: 2013-10-02 Last updated: 2018-01-11Bibliographically approved
Sohaib, A. T., Qureshi, S., Hagelbäck, J., Hilborn, O. & Jerčić, P. (2013). Evaluating classifiers for emotion recognition using EEG. In: : . Paper presented at 7th International Conference on Foundations of Augmented Cognition. Las Vegas: Springer
Open this publication in new window or tab >>Evaluating classifiers for emotion recognition using EEG
Show others...
2013 (English)Conference paper, Published paper (Refereed)
Abstract [en]

There are several ways of recording psychophysiology data from humans, for example Galvanic Skin Response (GSR), Electromyography (EMG), Electrocardiogram (ECG) and Electroencephalography (EEG). In this paper we focus on emotion detection using EEG. Various machine learning techniques can be used on the recorded EEG data to classify emotional states. K-Nearest Neighbor (KNN), Bayesian Network (BN), Artificial Neural Network (ANN) and Support Vector Machine (SVM) are some machine learning techniques that previously have been used to classify EEG data in various experiments. Five different machine learning techniques were evaluated in this paper, classifying EEG data associated with specific affective/emotional states. The emotions were elicited in the subjects using pictures from the International Affective Picture System (IAPS) database. The raw EEG data were processed to remove artifacts and a number of features were selected as input to the classifiers. The results showed that it is difficult to train a classifier to be accurate over large datasets (15 subjects) but KNN and SVM with the proposed features were reasonably accurate over smaller datasets (5 subjects) identifying the emotional states with an accuracy up to 77.78%.

Place, publisher, year, edition, pages
Las Vegas: Springer, 2013
Keywords
Emotion detection, Emotion recognition, Emotional state, Galvanic skin response, K nearest neighbor (KNN), Large datasets, Machine learning techniques, Picture system
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-6852 (URN)10.1007/978-3-642-39454-6_53 (DOI)oai:bth.se:forskinfo4E725BBDBF0A9820C1257C2E003674CF (Local ID)9783642394539 (ISBN)oai:bth.se:forskinfo4E725BBDBF0A9820C1257C2E003674CF (Archive number)oai:bth.se:forskinfo4E725BBDBF0A9820C1257C2E003674CF (OAI)
Conference
7th International Conference on Foundations of Augmented Cognition
Note

Held as Part of 15th International Conference on Human-Computer Interaction, HCI International 2013

Available from: 2013-11-25 Created: 2013-11-25 Last updated: 2018-01-11Bibliographically approved
Organisations

Search in DiVA

Show all publications