Change search
Link to record
Permanent link

Direct link
BETA
Jerčić, Petar
Alternative names
Publications (10 of 17) Show all publications
Jerčić, P., Hagelbäck, J. & Lindley, C. (2019). An affective serious game for collaboration between humans and robots. Entertainment Computing, 32, Article ID 100319.
Open this publication in new window or tab >>An affective serious game for collaboration between humans and robots
2019 (English)In: Entertainment Computing, ISSN 1875-9521, E-ISSN 1875-953X, Vol. 32, article id 100319Article in journal (Refereed) Published
Abstract [en]

Elicited physiological affect in humans collaborating with their robot partners was investigated to determine its influence on decision-making performance in serious games. A turn-taking version of the Tower of Hanoi game was used, where physiological arousal and valence underlying such human-robot proximate collaboration were investigated. A comparable decision performance in the serious game was found between human and non-humanoid robot arm collaborator conditions, while higher physiological affect was found in humans collaborating with such robot collaborators. It is suggested that serious games which are carefully designed to take into consideration the elicited physiological arousal might witness a better decision-making performance and more positive valence using non-humanoid robot partners instead of human ones. © 2019 The Authors

Place, publisher, year, edition, pages
Elsevier B.V., 2019
Keywords
Affect, Autonomous robots, Collaborative play, ECG, Emotions, GSR, Physiology, Robot-assisted play, Serious games, Anthropomorphic robots, Decision making, Electrocardiography, Robotic arms, Robots, Decision performance, Humanoid robot, Humanoid robot arm, Robot Partners, Tower of Hanoi
National Category
Computer Sciences
Identifiers
urn:nbn:se:bth-18739 (URN)10.1016/j.entcom.2019.100319 (DOI)000504663900006 ()
Note

open access

Available from: 2019-10-09 Created: 2019-10-09 Last updated: 2020-01-23
Jerčić, P. (2019). Arousal Measurement Reflected in the Pupil Diameter for a Decision-Making Performance in Serious Games. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics): . Paper presented at 1st IFIP TC 14 Joint International Conference on Entertainment Computing and Serious Games, ICEC-JCSG; Arequipa; Peru; 11 November 2019 through 15 November (pp. 287-298). Springer, 11863
Open this publication in new window or tab >>Arousal Measurement Reflected in the Pupil Diameter for a Decision-Making Performance in Serious Games
2019 (English)In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Springer , 2019, Vol. 11863, p. 287-298Conference paper, Published paper (Refereed)
Abstract [en]

This paper sets out to investigate the potentials of using pupil diameter measure as a contactless biofeedback method. The investigation was performed on how the interdependent and competing activation of the autonomic nervous system is reflected in the pupil diameter and how it affects the performance on decision-making task in serious games. The on-line biofeedback based on physiological measurements of arousal was integrated into the serious game set in the financial context. The pupil diameter was validated against the heart rate data measuring arousal, where the effects of such arousal were investigated. It was found that the physiological arousal was observable on both the heart and pupil data. Furthermore, the participants with lower arousal took less time to reach their decisions, and those decisions were more successful, in comparison to the participants with higher arousal. Moreover, such participants were able to get a higher total score and finish the game. This study validated the potential usage of pupil diameter as an unobtrusive measure of biofeedback, which would be beneficial for the investigation of arousal on human decision-making inside of serious games. © IFIP International Federation for Information Processing, 2019.

Place, publisher, year, edition, pages
Springer, 2019
Keywords
Arousal, Decision-making, Heart-rate variability, Physiology, Pupil diameter, Serious games, Biofeedback, Decision making, Heart, Neurophysiology, Autonomic nervous system, Contact less, Data measuring, Heart rate variability, Human decision making, Physiological measurement
National Category
Computer Sciences
Identifiers
urn:nbn:se:bth-19086 (URN)10.1007/978-3-030-34644-7_23 (DOI)2-s2.0-85076953084 (Scopus ID)9783030346430 (ISBN)
Conference
1st IFIP TC 14 Joint International Conference on Entertainment Computing and Serious Games, ICEC-JCSG; Arequipa; Peru; 11 November 2019 through 15 November
Available from: 2020-01-09 Created: 2020-01-09 Last updated: 2020-01-09Bibliographically approved
Jerčić, P. & Sundstedt, V. (2019). Practicing Emotion-Regulation Through Biofeedback on the Decision-Making Performance in the Context of Serious Games: a Systematic Review. Entertainment Computing, 29, 75-86
Open this publication in new window or tab >>Practicing Emotion-Regulation Through Biofeedback on the Decision-Making Performance in the Context of Serious Games: a Systematic Review
2019 (English)In: Entertainment Computing, ISSN 1875-9521, E-ISSN 1875-953X, Vol. 29, p. 75-86Article in journal (Refereed) Published
Abstract [en]

Evidence shows that emotions critically influence human decision-making. Therefore, emotion-regulation using biofeedback has been extensively investigated. Nevertheless, serious games have emerged as a valuable tool for such investigations set in the decision-making context. This review sets out to investigate the scientific evidence regarding the effects of practicing emotion-regulation through biofeedback on the decision-making performance in the context of serious games. A systematic search of five electronic databases (Scopus, Web of Science, IEEE, PubMed Central, Science Direct), followed by the author and snowballing investigation, was conducted from a publication's year of inception to October 2018. The search identified 16 randomized controlled experiment/quasi-experiment studies that quantitatively assessed the performance on decision-making tasks in serious games, involving students, military, and brain-injured participants. It was found that the participants who raised awareness of emotions and increased the skill of emotion-regulation were able to successfully regulate their arousal, which resulted in better decision performance, reaction time, and attention scores on the decision-making tasks. It is suggested that serious games provide an effective platform validated through the evaluative and playtesting studies, that supports the acquisition of the emotion-regulation skill through the direct (visual) and indirect (gameplay) biofeedback presentation on decision-making tasks.

Place, publisher, year, edition, pages
Elsevier, 2019
Keywords
serious games, emotions, affect, biofeedback, emotion-regulation, decision-making
National Category
Computer Systems
Identifiers
urn:nbn:se:bth-17580 (URN)10.1016/j.entcom.2019.01.001 (DOI)000458213700008 ()
Note

open access

Available from: 2019-02-05 Created: 2019-02-05 Last updated: 2019-03-04Bibliographically approved
Jerčić, P. (2019). The Effects of Emotions and Their Regulation on Decision-making Performance in Affective Serious Games. (Doctoral dissertation). Karlskrona: Blekinge Tekniska Högskola
Open this publication in new window or tab >>The Effects of Emotions and Their Regulation on Decision-making Performance in Affective Serious Games
2019 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Emotions are thought to be one of the key factors that critically influence human decision-making. Emotion-regulation can help to mitigate emotion-related decision biases and eventually lead to a better decision performance. Serious games emerged as a new angle introducing technological methods to practicing emotion-regulation, where meaningful biofeedback information communicates player's affective states to a series of informed gameplay choices. These findings motivate the notion that in the decision context of serious games, one would benefit from awareness and regulation of such emerging emotions.

This thesis explores the design and evaluation methods for creating serious games where emotion-regulation can be practiced using physiological biofeedback measures. Furthermore, it investigates emotions and the effect of emotion-regulation on decision performance in serious games. Using the psychophysiological methods in the design of such games, emotions and their underlying neural mechanism have been explored.

The results showed the benefits of practicing emotion-regulation in serious games, where decision-making performance was increased for the individuals who down-regulated high levels of arousal while having an experience of positive valence. Moreover, it increased also for the individuals who received the necessary biofeedback information. The results also suggested that emotion-regulation strategies (i.e., cognitive reappraisal) are highly dependent on the serious game context. Therefore, the reappraisal strategy was shown to benefit the decision-making tasks investigated in this thesis. The results further suggested that using psychophysiological methods in emotionally arousing serious games, the interplay between sympathetic and parasympathetic pathways could be mapped through the underlying emotions which activate those two pathways. Following this conjecture, the results identified the optimal arousal level for increased performance of an individual on a decision-making task, by carefully balancing the activation of those two pathways. The investigations also validated these findings in the collaborative serious game context, where the robot collaborators were found to elicit diverse affect in their human partners, influencing performance on a decision-making task. Furthermore, the evidence suggested that arousal is equally or more important than valence for the decision-making performance, but once optimal arousal has been reached, a further increase in performance may be achieved by regulating valence. Furthermore, the results showed that serious games designed in this thesis elicited high physiological arousal and positive valence. This makes them suitable as research platforms for the investigation of how these emotions influence the activation of sympathetic and parasympathetic pathways and influence performance on a decision-making task.

Taking these findings into consideration, the serious games designed in this thesis allowed for the training of cognitive reappraisal emotion-regulation strategy on the decision-making tasks. This thesis suggests that using evaluated design and development methods, it is possible to design and develop serious games that provide a helpful environment where individuals could practice emotion-regulation through raising awareness of emotions, and subsequently improve their decision-making performance.

Place, publisher, year, edition, pages
Karlskrona: Blekinge Tekniska Högskola, 2019. p. 297
Series
Blekinge Institute of Technology Doctoral Dissertation Series, ISSN 1653-2090 ; 6
Keywords
Serious Games, Game Design, Emotions, Biofeedback, Emotion-Regulation, Decision-Making
National Category
Computer Systems
Identifiers
urn:nbn:se:bth-17557 (URN)978-91-7295-370-3 (ISBN)
Public defence
2019-04-26, J1640, Campus Gräsvik, Karlskrona, 09:12 (English)
Opponent
Supervisors
Available from: 2019-01-31 Created: 2019-01-30 Last updated: 2019-05-09Bibliographically approved
Jerčić, P. (2019). What could the baseline measurements predict about decision-making performance in serious games set in the financial context. In: 2019 11th International Conference on Virtual Worlds and Games for Serious Applications, VS-Games 2019 - Proceedings: . Paper presented at 11th International Conference on Virtual Worlds and Games for Serious Applications, VS-Games, Vienna, 4 September through 6 September. Institute of Electrical and Electronics Engineers Inc.
Open this publication in new window or tab >>What could the baseline measurements predict about decision-making performance in serious games set in the financial context
2019 (English)In: 2019 11th International Conference on Virtual Worlds and Games for Serious Applications, VS-Games 2019 - Proceedings, Institute of Electrical and Electronics Engineers Inc. , 2019Conference paper, Published paper (Refereed)
Abstract [en]

This paper sets out to investigate how the basal activation of the parasympathetic and sympathetic nervous systems may affect and predict the decision-making performance of players in serious games. In order to investigate the basal activation of both branches of the autonomic nervous system, pupil diameter and heart rate were recorded during baseline and analyzed in regards to performance scores in the serious game. It was found that the balance between the parasympathetic and sympathetic activation was responsible for beneficial decision-making performance, while lower sympathetic activation was found to be associated with the higher level reached in the game. It is suggested that the balance between the basal activation of both branches of the autonomic nervous system recorded during the baseline may predict the decision-making performance of players on the subsequent tasks in serious games. © 2019 IEEE.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers Inc., 2019
Keywords
Decision-making, Parasympathetic nervous system, Physiology, Pupil diameter, Serious games, Sympathetic nervous system, Chemical activation, Decision making, Forecasting, Virtual reality, Autonomic nervous system, Heart rates, Parasympathetic nervous systems, Sympathetic nervous systems
National Category
Human Computer Interaction
Identifiers
urn:nbn:se:bth-18920 (URN)10.1109/VS-Games.2019.8864586 (DOI)2-s2.0-85074289316 (Scopus ID)9781538671238 (ISBN)
Conference
11th International Conference on Virtual Worlds and Games for Serious Applications, VS-Games, Vienna, 4 September through 6 September
Available from: 2019-11-18 Created: 2019-11-18 Last updated: 2019-11-18Bibliographically approved
Horvat, M., Dobrinic, M., Novosel, M. & Jerčić, P. (2018). Assessing emotional responses induced in virtual reality using a consumer EEG headset: A preliminary report. In: 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics, MIPRO 2018 - Proceedings: . Paper presented at 41st International Convention on Information and Communication Technology, Electronics and Microelectronics, MIPRO , Opatija, Croatia (pp. 1006-1010). Institute of Electrical and Electronics Engineers Inc.
Open this publication in new window or tab >>Assessing emotional responses induced in virtual reality using a consumer EEG headset: A preliminary report
2018 (English)In: 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics, MIPRO 2018 - Proceedings, Institute of Electrical and Electronics Engineers Inc. , 2018, p. 1006-1010Conference paper, Published paper (Refereed)
Abstract [en]

We report on a pilot study involving emotion elicitation in virtual reality (VR) and assessment of emotional responses with a consumer-grade EEG device. The stimulation used HTC Vive VR system showing pictures from NAPS database within a specifically designed virtual environment. The stimulation consisted of two distinct sequences with 10 pictures of happiness and 10 pictures of fear. Each picture was contained in a separate virtual room that the participants traveled through along a preset path. The estimation employed EMOTIV EPOC+ 14-channel EEG headset and a custom-developed application. The software wirelessly received EEG signals from alpha, beta low, beta high, gamma and theta bands, time-stamped them and dynamically stored in a relational database for subsequent analysis. Our preliminary results show that statistically significant correlations between valence and arousal ratings of pictures and EEG bands are present but highly personalized. Simultaneous correct placement of VR and EEG headsets is demanding and precise localization of electrodes is difficult. In fact, if emotion estimation is not strictly necessary we recommend using devices with fewer electrodes. Nevertheless, we found the EEG to be effective. By acknowledging its limitations, and using the headset in the correct context, experiments involving emotions may be significantly amended. © 2018 Croatian Society MIPRO.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers Inc., 2018
Keywords
Electrodes, Microelectronics, Developed applications, Emotion elicitation, Emotion estimation, Emotional response, Emotiv epoc, Pilot studies, Relational Database, Virtual rooms, Virtual reality
National Category
Other Electrical Engineering, Electronic Engineering, Information Engineering
Identifiers
urn:nbn:se:bth-16908 (URN)10.23919/MIPRO.2018.8400184 (DOI)2-s2.0-85050249600 (Scopus ID)9789532330977 (ISBN)
Conference
41st International Convention on Information and Communication Technology, Electronics and Microelectronics, MIPRO , Opatija, Croatia
Available from: 2018-08-20 Created: 2018-08-20 Last updated: 2018-08-20Bibliographically approved
Jerčić, P., Sennersten, C. & Lindley, C. (2018). Modeling cognitive load and physiological arousal through pupil diameter and heart rate. Multimedia tools and applications
Open this publication in new window or tab >>Modeling cognitive load and physiological arousal through pupil diameter and heart rate
2018 (English)In: Multimedia tools and applications, ISSN 1380-7501, E-ISSN 1573-7721Article in journal (Refereed) Epub ahead of print
Abstract [en]

This study investigates individuals’ cognitive load processing abilities while engaged on a decision-making task in serious games, to explore how a substantial cognitive load dominates over the physiological arousal effect on pupil diameter. A serious game was presented to the participants, which displayed the on–line biofeedback based on physiological measurements of arousal. In such dynamic decision-making environment, the pupil diameter was analyzed in relation to the heart rate, to evaluate if the former could be a useful measure of cognitive abilities of individuals. As pupil might reflect both cognitive activity and physiological arousal, the pupillary response will show an arousal effect only when the cognitive demands of the situation are minimal. Evidence shows that in a situation where a substantial level of cognitive activity is required, only that activity will be observable on the pupil diameter, dominating over the physiological arousal effect indicated by the pupillary response. It is suggested that it might be possible to design serious games tailored to the cognitive abilities of an individual player, using the proposed physiological measurements to observe the moment when such dominance occurs. © 2018, The Author(s).

Place, publisher, year, edition, pages
Springer New York LLC, 2018
Keywords
Arousal, Cognitive load, Electrocardiogram, Physiology, Pupil diameter, Serious games, Biofeedback, Decision making, Electrocardiography, Heart, Neurophysiology, Physiological models, Cognitive activities, Cognitive loads, Dynamic decision making, Physiological measurement, Processing ability, Pupillary response, Psychophysiology
National Category
Other Computer and Information Science
Identifiers
urn:nbn:se:bth-17052 (URN)10.1007/s11042-018-6518-z (DOI)2-s2.0-85053407843 (Scopus ID)
Note

open access

Available from: 2018-09-27 Created: 2018-09-27 Last updated: 2019-01-30Bibliographically approved
Jerčić, P., Hagelbäck, J. & Lindley, C. (2018). Physiological Affect and Performance in a Collaborative Serious Game Between Humans and an Autonomous Robot. In: Lect. Notes Comput. Sci.: . Paper presented at 17th IFIP TC 14 International Conference on Entertainment Computing, ICEC, Poznan (pp. 127-138). Springer Verlag, 11112
Open this publication in new window or tab >>Physiological Affect and Performance in a Collaborative Serious Game Between Humans and an Autonomous Robot
2018 (English)In: Lect. Notes Comput. Sci., Springer Verlag , 2018, Vol. 11112, p. 127-138Conference paper, Published paper (Refereed)
Abstract [en]

This paper sets out to examine how elicited physiological affect influences the performance of human participants collaborating with the robot partners on a shared serious game task; furthermore, to investigate physiological affect underlying such human-robot proximate collaboration. The participants collaboratively played a turn-taking version of a serious game Tower of Hanoi, where physiological affect was investigated in a valence-arousal space. The arousal was inferred from the galvanic skin response data, while the valence was inferred from the electrocardiography data. It was found that the robot collaborators elicited a higher physiological affect in regard to both arousal and valence, in contrast to their human collaborator counterparts. Furthermore, a comparable performance between all collaborators was found on the serious game task. © 2018, IFIP International Federation for Information Processing.

Place, publisher, year, edition, pages
Springer Verlag, 2018
Series
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), ISSN 03029743
Keywords
Affect, Autonomous robots, Collaborative play, Emotions, Physiology, Robot-assisted play, Serious games, Electrophysiology, Robots, Galvanic skin response, Human robots, Robot Partners, Tower of Hanoi, Turn-taking
National Category
Computer Sciences
Identifiers
urn:nbn:se:bth-17087 (URN)10.1007/978-3-319-99426-0_11 (DOI)2-s2.0-85053772889 (Scopus ID)9783319994253 (ISBN)
Conference
17th IFIP TC 14 International Conference on Entertainment Computing, ICEC, Poznan
Available from: 2018-10-05 Created: 2018-10-05 Last updated: 2019-01-30Bibliographically approved
Jerčić, P., Wen, W., Hagelbäck, J. & Sundstedt, V. (2018). The Effect of Emotions and Social Behavior on Performance in a Collaborative Serious Game Between Humans and Autonomous Robots. International Journal of Social Robotics, 10(1), 115-129
Open this publication in new window or tab >>The Effect of Emotions and Social Behavior on Performance in a Collaborative Serious Game Between Humans and Autonomous Robots
2018 (English)In: International Journal of Social Robotics, ISSN 1875-4791, E-ISSN 1875-4805, Vol. 10, no 1, p. 115-129Article in journal (Refereed) Published
Abstract [en]

The aim of this paper is to investigate performance in a collaborative human–robot interaction on a shared serious game task. Furthermore, the effect of elicited emotions and perceived social behavior categories on players’ performance will be investigated. The participants collaboratively played a turn-taking version of the Tower of Hanoi serious game, together with the human and robot collaborators. The elicited emotions were analyzed in regards to the arousal and valence variables, computed from the Geneva Emotion Wheel questionnaire. Moreover, the perceived social behavior categories were obtained from analyzing and grouping replies to the Interactive Experiences and Trust and Respect questionnaires. It was found that the results did not show a statistically significant difference in participants’ performance between the human or robot collaborators. Moreover, all of the collaborators elicited similar emotions, where the human collaborator was perceived as more credible and socially present than the robot one. It is suggested that using robot collaborators might be as efficient as using human ones, in the context of serious game collaborative tasks.

Place, publisher, year, edition, pages
Springer, 2018
Keywords
Autonomous robots, Serious games, Collaborative play, Social interaction, Robot-assisted play, Emotions
National Category
Computer Systems
Identifiers
urn:nbn:se:bth-15541 (URN)10.1007/s12369-017-0437-4 (DOI)000423152900008 ()
Projects
PsyIntEC
Funder
EU, FP7, Seventh Framework Programme, FP7-ICT-231143
Note

open access

Available from: 2017-11-28 Created: 2017-11-28 Last updated: 2019-01-30Bibliographically approved
Jerčić, P., Sennersten, C. & Lindley, C. (2017). The effect of cognitive load on physiological arousal in a decision-making serious game. In: 2017 9th International Conference on Virtual Worlds and Games for Serious Applications, VS-Games 2017 - Proceedings: . Paper presented at 9th International Conference on Virtual Worlds and Games for Serious Applications, VS-Games, Athens (pp. 153-156). Institute of Electrical and Electronics Engineers Inc., Article ID 8056587.
Open this publication in new window or tab >>The effect of cognitive load on physiological arousal in a decision-making serious game
2017 (English)In: 2017 9th International Conference on Virtual Worlds and Games for Serious Applications, VS-Games 2017 - Proceedings, Institute of Electrical and Electronics Engineers Inc. , 2017, p. 153-156, article id 8056587Conference paper, Published paper (Refereed)
Abstract [en]

The aim of this paper is to investigate how a substantial cognitive load overshadows the physiological arousal effect, in an attempt to study cognitive abilities of participants engaged on decision-making tasks in serious games. Participants were engaged in a dynamic serious game environment displaying online biofeedback based on the physiological measurements of arousal. The pupil diameter was analyzed in relation to the heart rate during a challenging decision-making task. It was found that the moment when a substantial cognitive load overshadows the physiological arousal effect is observable on the pupil diameter in relation to the heart rate. © 2017 IEEE.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers Inc., 2017
Series
International Conference on Games and Virtual Worlds for Serious Applications, ISSN 2474-0470
Keywords
arousal, cognitive load, electrocardiogram, physiology, pupil diameter, serious games, Biofeedback, Decision making, Electrocardiography, Heart, Neurophysiology, Psychophysiology, Virtual reality, Cognitive ability, Cognitive loads, Game environment, Heart rates, Physiological measurement
National Category
Human Computer Interaction
Identifiers
urn:nbn:se:bth-15606 (URN)10.1109/VS-GAMES.2017.8056587 (DOI)000425228700023 ()2-s2.0-85034631890 (Scopus ID)9781509058129 (ISBN)
Conference
9th International Conference on Virtual Worlds and Games for Serious Applications, VS-Games, Athens
Available from: 2017-12-14 Created: 2017-12-14 Last updated: 2018-10-19Bibliographically approved
Organisations

Search in DiVA

Show all publications