Emotional Annotation of Text David Gallagher
Download 36.25 Kb.
|
- Navigate this page:
- Introduction
- Framework and Background
- Datasets Five datasets were used in the experiment by Chaffar and Inkpen, these are detailed below. Text Affect
- Neviarouskaya et al.’s Dataset
- Alm’s Dataset
- Aman’s Dataset
- Feature Sets
- Table 2.
Emotional Annotation of Text David Gallagher Department of Computer Science University of Wisconsin-Platteville GallagherD@uwplatt.edu Abstract Emotions play an important role in human intelligence, rational decision making, social interaction, perception, memory, learning, creativity, and more [4]. Automatic detection of emotions in texts is progressively becoming an area of research demanding attention, since emotions convey a vast amount of information that is difficult to comprehend and often difficult to express in words, particularly in text. Effective analysis of text can lead to a vast array of applications, such as opinion mining, market analysis, affective computing, and natural language interfaces such as e-learning environments or educational/edutainment games. Emotions have been deeply studied in order to gain a better understanding of human interaction. While the subject of emotions has been classically researched by the fields of psychology and behavior sciences, with increasingly integrated communications, the subject is applicable to the field of computer science, particularly in field of human computer interaction. Introduction Emotions have been a classical area of study for various disciplines such as psychology and behavior sciences, as emotions play an intrinsic role in our human nature. In particular, emotions convey a vast amount of subtext or connotation that can greatly change the context of the communications that individuals share with one another. Researchers in the field of computer science have carried out various studies on the emotions represented by facial expression [6] and the recognition of these emotions, through a variety of sensors analyzing the various facial expressions shown in the various emotions [3]. The most natural way for a computer to recognize the emotion of the user is to detect his or her emotional state from the text that the user entered be it from a blog, online chat site, or in another form of text [2]. The automatic emotional annotation in texts is becoming increasing important from an applicative point of view, for the advancement of many related fields. Affective computing, or natural language interfaces such as e-learning environments or education/edutainment games would benefit significantly from automated emotional annotation of text. The study would also greatly increase the effectiveness of machine-driven tasks of opinion mining and market analysis. For example, the following are specific areas that the application of an automated affective analysis could make intriguing and invaluable advancements:
Knowledge-based approaches and machine learning approaches were adopted for automatic analysis of emotions in text, aiming to detect the writer’s emotional state. Knowledge-based approaches consist of using linguistic models or prior knowledge to classify emotional text. Machine learning approaches use supervised learning algorithms to build models from annotated corpora or a large and structured set of texts. Research done in the field of sentiment analysis have also been applied various linguistic models and different learning algorithms. The machine learning technique tended to perform better than lexical-based techniques because they can adapt well to different domains [2]. This paper highlights affective analysis background and related fields and a simplified process to conduct computer assisted emotional annotation of text. Framework and Background Deriving the emotional content of a text through linguistic analysis is an extremely, even infamously difficult task. Many fields, such as psychology, sociology, and philosophy, have proposed approaches for the emotion detection. These fields have studied emotions with respect to facial expressions, action tendencies, physiological activity, and subjective experience [3]. A text-based emotion prediction system would benefit from identifying the emotional affinity of sentences. The emotion analysis on sentence level may also be important for more detailed emotion analysis systems [5]. Several researchers have attempted to solve this issue in distinct ways. Cecilia Ovesdotter Alm a professor at Rochester Institute of Technology explored the text-based emotion prediction problem. In order to classify the emotional affinity of sentences in the narrative domain of children’s fairy tales, they annotated a corpus of 22 Grimms’ tales on sentence level with eight emotion categories (angry, disgusted, fearful, happy, sad, positively surprised, and negatively surprised). Alena Neviarouskaya, a JSPS Postdoctoral Researcher in the Knowledge Data Engineering and Information Retrieval Laboratory, Department of Computer Science and Engineering, Toyohashi University of Technology, addressed the tasks of recognition and interpretation of affect communicated through text messaging [1]. Classifying the mood of a single text is a hard task; state-of-the-art methods in text classification achieve only modest performance in this domain. In this area, some of the hardest problems involve acquiring large collection of text tagged with detail linguistic expressions that indicate emotion. Development To lay the framework for the development of automatic emotional annotation one must decide what are the most basic of emotions. This is an important step and there are several emotional models that have been developed that may be used as a pivotal resource. These emotional models include:
Figure 1. Plutchik’s Wheel 
Even with a model that can accurately annotate text with values of the chosen basic emotions, several emotions may have similar or ambiguous meaning such as happy and contented. A technique to distinguish these words is through the use of emotional dimensions. There are three categories of emotional dimension: evaluation, activation and power. Evaluation represents how positive or negative an emotion is. Activation represents an active or passive scale for emotions. Power represents the control that is exerted, at one end of the scale we have emotions that are submissive and at the other end we have emotions that are dominant. As this paper has alluded, there are many ways to develop an effective automatic emotional annotation algorithm. To highlight one approach we will analyze the Sequential Minimal Optimization (SMO) implementation of the Support Vector Machine (SVM) illustrated by Soumaya Chaffar a researcher and Diana Inkpen a professor of computer science at the University of Toronto. One must develop a dataset or emotional dictionary that can be used for the emotion look-up detection in text. It is useful to have a variety of datasets collected in from different sources as one may be better suited for a different type of development. Next in order to further analyze the sentence, a feature set can be applied in order to highlight specific things such as negative words, conjunctions, punctuations, contexts and so on. At this point an algorithm may be applied to derive the emotional annotation of a text. Finally the algorithm can be compared to analyze its ability to distinguish emotions. Datasets Five datasets were used in the experiment by Chaffar and Inkpen, these are detailed below. Text Affect This dataset consisted two separate parts drawn from news headlines from renowned newspapers, as well from the Google News search engine. The first part was developed for the training and composed of 250 annotated sentences. The second part was designed for testing and it consisted of 1,000 annotated sentences. Six emotions (anger, disgust, fear, joy, sadness and surprise - similar to the Ekman model) were used to annotate sentences according to the degree of emotional load. Neviarouskaya et al.’s Dataset This data set was developed by Neviarouskaya and others. In these datasets, ten labels were utilized to annotate sentences by three annotators. These labels consist of the nine emotional categories defined by Izard; anger, disgust, fear, guilt, interest, joy, sadness, shame, surprise and a neutral category. For their experiment Chaffar and Inkpen only considered sentences on which two annotators or more completely agreed on the emotion category.
Alm’s Dataset Alm’s Dataset contained annotated sentences from fairy tales - Grimm’s Fairy Tales. In the highlighted experiment only sentences with high emotional agreement were used in the experiment. Ekman’s list of basic emotions was used for sentences annotations, because of data sparsely and related semantics between anger and disgust, these two emotions were merged together by Alm. This resulted in the five emotions of happy, fearful, sad, surprised and angry-disgusted. Aman’s Dataset This dataset consists of emotion-rich sentences collected from blogs. Ekman’s basic emotions happiness, sadness, anger, disgust, surprise, fear and also a neutral category were used for sentences annotation. The sentences were labeled with emotions by four annotators. The experiment considered only sentences for which the annotators agreed on the emotion category. Feature Sets Feature sets can be applied in order to highlight specific things such as negative words, conjunctions, punctuations, contexts, which could drastically change the meaning and emotional load of a sentence. To ensure proper emotional classification of text, it is essential to choose the relevant feature sets to be considered. Various feature sets are illuminated below:
Table 1. MiniPar example of dependency tree for the sentence “two of her tears wetted his eyes and they grew clear again 
Table 2. Fragment of a marked up table. 
Algorithm Application Weka (Waikato Environment for Knowledge Analysis) is a popular suite of machine learning software written in Java, developed at the University of Waikato, New Zealand. Using various Weka software tools Chaffar and Inkpen’s experiment illustrates the effectiveness of different algorithms on the various datasets discussed earlier; J48 for Decision Trees, Naïve Bayes for a Bayesian classifier and the SMO implementation of SVM. The Weka ZeroR classifier was used as a base line because this classifier does not take into account any sort of feature set (See Table 3). Table 3. Results for the training datasets using the accuracy rate (%) 
From Table 3 we can see that the SMO algorithm distinguishes itself and the premier algorithm in this selection, as it has the highest accuracy for matching emotions. In the next section of their experiment they applied the SMO algorithm with various feature sets to see the highest accuracy. In this section they used different data sets to generalize unseen examples (See Table 4). 
It is interesting to note that for the various datasets the simplified BOW approach seems to achieve the highest accuracy on most of the test sets/datasets. This could be explained by the fact that the SMO algorithm does not accurately account for the various features presented by the feature sets, the feature sets themselves contain some type of fundamental error or the test sets/datasets presented do not accurately meet the expectations of the algorithm. Conclusions Written language is one of our most common forms of communication and only increasing in popularity, besides transmitting informative content, it also transmits information about the user’s attitude including the user’s emotional state. While there have been many studies carried out in the field of human computer interaction, comparatively little research has been devoted to the detection of emotions in texts [6]. There is a smorgasbord of related fields that could benefit from and contribute to the advancement of automated emotional annotation of text. Before that can happen further research and work is needed to aid the advancement of this field and aid the proper development of the semantic web which could be used to boost the development of the emotional annotation of text. References [1] Alm, Roth and Sproat. “Emotions from Text: Machine Learning for Text-based Emotion Prediction.” Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, Vancouver, British Columbia, Canada, pp.579-586, 2005. [2] Chaffar, Soumaya, and Diana Inkpen. "Using a Heterogeneous Dataset for Emotion Analysis in Text." School of Information Technology and Engineering, University of Ottawa Ottawa, ON, Canada. University of Ottawa, 2011. Web. 7 Oct 2012. [3] Devillers, Laurence , Laurence Vidrascu, and Lori Lamel. "Challenges in real-life emotion annotation and machine learning based detection." Neural Networks. 18.4 (2005): 407-422, ISSN 0893-6080. [4] Picard, R. W. “Affective Computing.” The MIT Press, MA, USA, 1997. [5] Quan, Changqin, and Fuji Ren. "Sentence Emotion Analysis and Recognition Based on Emotion Words Using Ren-CECps." International Journal of Advanced Intelligence. 2.1 (2010): 105-117. Web. 7 Oct. 2012. [6] Strapparava , Carlo, and Rada Mihalcea. "Learning to Identify Emotions in Text". Fortaleza, Brazile: 2008. Web. 7 Oct. 2012. Directory: ~yangq -> csse411 -> csse411-materials -> f12 csse411-materials -> Multi-Core Software Development with Examples in C++ csse411-materials -> Web Security a Programmer’s Perspective f12 -> Christopher Pinski Department of Computer Science and Software Engineering csse411-materials -> Computer Science / Software Engineering uw-platteville csse411-materials -> Security of Mobile Devices Lon Kastenson Abstract csse411-materials -> Developing for Android Erik Nykl Abstract csse411-materials -> Spyware, Malware, and Viruses Anthony Bosnak Department of Computer Science & Software Engineering University of Wisconsin – Platteville csse411-materials -> Personal Computing: Past, Present, and Future Michael Wills csse411-materials -> Virtual Reality (VR) Kyle Maitland Department of Computer Science University of Wisconsin Platteville Download 36.25 Kb. Share with your friends:
|