The proposed system continuously tracks the kinematics of the body expressions and classifies them according to four affective classes. Using a majority rule, an overall affective class is associated with each game-point window. The results obtained are interesting as they are well above chance level and comparable with other automatic recognition systems of naturalistic affective expressions for other modalities (e.g., winners of the AVEC’11 challenge [70], [71]). In this section, we discuss the possible use of these results and issues that need to be taken into account when using such information to measure players’ aesthetic experience.
A.Modulating the aesthetic experience
Whereas we do not claim that the detected affective expressions do always represent what the players are feeling in that moment, we argue that the affective expressions that the players unconsciously portray through their body movement have a strong bias on their experience through proprioceptive feedback, as discussed in the literature review. Thus, detecting such affective expressions provides insight into both the affective states the person may be in and his/her readability to shift from a certain type of affective experience to another. As a matter of fact, what the system does is to recognize the variety of emotions the player expresses in the course of an aesthetic experience. In pragmatist terms, such a variety can constitute the emotional rhythmic dynamics of aesthetic experience culminating in the unifying underlying felt quality of an aesthetic experience. In other words, the emotions conveyed through the body may express the readiness of the player to enter the movement-affection loop proposed in [6] and reach the ‘closure’ of an aesthetic experience in a full-body game scenario. In her studies in [6], Bianchi-Berthouze showed that role-play body movements imposed on the player (e.g., raising the guitar neck to acquire more points during a Guitar Hero game session) and their expressive quality (the way in which this movement is performed) brought the player to enter the movement-affective loop within which a larger variation of emotions and gestures related to the role-play situation of the game (e.g., a rock music player: more positive emotions, dancing, rock player gestures) were expressed, even if they were interfering with the goal of winning the game.
Here we are suggesting that the occurring patterns of expressions could be used to infer the desire or the readiness to reach a closure and hence the fulfilment of an aesthetic experience. Further improvements in the recognition performances are necessary before such on-line adaptation could be performed.
B.Factors to be accounted for
One of the questions to be asked at this point is how long does an affective body expression need to be expressed in order to have an effect on cognitive and affective processes, i.e., in modulating the aesthetic experience of the player? In our study, two assumptions were made. First, on the basis of experimental results, the length of the intervals to use as a unit of expression in the game-point was set to 10 frames (i.e., 167 ms). It needs to be considered however that these 10-frames windows overlapped. Second, it was here considered that the most frequent type of detected affective expressions would most modulate the player’s appraisal of the game experience. Is the duration of these expressions sufficient?
Unfortunately, the studies reported in the literature (see section II.A) are quite vague about the duration of a posture/movement to have a modulatory effect. In [18], the inducing condition lasted the time necessary to read stories such as “Donald refuses to pay his rent, but only after his landlord failed to make promised repairs”. In [21], Stepper et al. speculate on this length. According to their results, the postural influence appears to be stronger at the onset of the proprioceptive feedback (i.e., as the posture is assumed) rather than after a prolonged maintenance of the position. They suggest that this could be due either to habituation as “effects of neural impingements are typically produced by changes in sensory input and not by its mere state”. This may indicate that, if a certain type of expression is maintained for a longer period, it may have a weaker impact. Hence, it could be expected that the most frequent affective type of expressions during a game point may have an effect on the players’ aesthetic experience. In [8], the use of aggressive expressions, even if not continuously portrayed, did appear to have a significant effect on the emotional state of the player.
To our knowledge, however, in all the studies presented in this literature, the expressions were imposed on, rather than being naturally expressed by, the participants. This may have implications for how the contribution of each body expression (imposed by the game designer or improvised by the player) should be used to measure the quality of an aesthetic experience. In his study, Riskind [72] provides evidence of the effect of posture on affect but it also highlights the fact that this effect is modulated by the congruence between the valence of the posture and the mood of the person (appropriateness hypothesis). It is also important to consider the context in which certain expressions occur. [20] also points out that “the joint occurrence of bodily changes and the activation of the appropriate cognitive contents are likely to provide the optimal conditions for a full-blown emotional experience”. It is possible that body movements may have a stronger effect when game events (but not body movement) are consciously appraised. Rather than a majority vote, therefore, a more complex weighting of these expressions and their alternation may need to be considered in the modelling.
Finally, players’ personalities may have an impact upon how people are affected by proprioceptive feedback. In our study, we labelled the data according to how people read the body expressions of the players. Whilst the emotion classes used in this study are considered universal, studies have shown that differences exist, especially in terms of how people express the intensity of an emotion [48]. It is, therefore, possible that, when building the training set for these systems, we consider personality, culture and gender of the expresser, as they may have an important role in the modulating process. In fact, in [18], the authors show that culturally-learned body movements also have an effect on our cognitive and affective processes.
Share with your friends: |