Semester: 10th Semester, Master Thesis Title

Download 0.97 Mb.
Size0.97 Mb.
  1   2   3   4   5   6   7   8   9   ...   19
Semester: 10th Semester, Master Thesislogo

Title: Software Based Affective Smile Estimation

Project Period: 020212-250512


The following Master Thesis was finished at Aalborg University Campus Ballerup during the 10th semester at Medialogy.

This thesis investigated if a computer could be programmed to interpret and reason the human smile with the same accuracy and understanding as that of humans. Research revealed that attempts at extracting meaning from facial features in Computer Vision had not been attempted from the point of view of the computer. Two testing phases were devised that provided an average smile rating based on answers given by test subjects. For test phase one the computer had a 50% success ratio of correctly rating the smile conveyed in pictures. In test phase two the computer rated the smile at the same level as the test subjects. The computer was able to understand and rate smiles that were conveyed with visual distinct features, whereas ubiquitous smiles could not be rated.

This thesis covered the following terms, Affective Computing, The Facial Action Coding System, Action Units and Emotional Intelligence. Understanding and rating a smile could only be conducted from visually distinct smiles, whereas when gaze was the predominant visual factor, the computer could not rate accordingly.

Aalborg University Copenhagen

Lautrupvang 15, 2750 Ballerup, Denmark

Semester Coordinator: Stefania Serafin 

Secretary: Judi Stærk Poulsen

Semester Theme: Master Thesis

Supervisor: Lars Reng


Tomas Fuchs Bøttern

Copies: 3

Pages: 108

Finished: 240512

Project group no.: Not Available

Copyright © 2012. This report and/or appended material may not be partly or completely published or copied without prior written approval from the author. Neither may the contents be used for commercial purposes without this written approval.


This Master Thesis was written by Tomas Fuchs Bøttern attending Aalborg University Copenhagen Denmark, as a result of the 10th semester at Medialogy.

This report uses ISO 690 when citing works, as this is the standard in bibliographic referencing. The parenthesis following a citation or written text i.e. (Author’s Last Name, Year) refers to a source used for that particular chapter.

The Bibliography chart also follows the design and rules laid forth in the ISO 690 standard, namely the layout referencing of published material both in its print and non-print form. The following is an example of the layout of an article reference.

Reference in the Thesis:

(Russel, et al., 1997)

In Bibliography:

Russel, James A and Fernández-Dols, José Miguel. 1997. The psychology of facial expression. Cambridge: The Press Syndicate of The University of Cambridge, 1997. 0 521 49667 5

Following this thesis is a DVD located at the back of the report containing the digital equivalent of the references and this thesis. The software used for establishing the smile rating is included.

On the attached DVD locate [DVD-DRIVE]:\TOC.txt. The Table of Contents (TOC.txt) contains the guide on how to use the software created for this thesis. Furthermore the TOC.txt lists all directories on the DVD as well as describing their content.

Lastly, this thesis wishes to thank the following persons / institutions for their assistance in the creation of this thesis:

Sønderborg Fjernvarme A.m.b.a. for their participation in Test Phase One.

Table of Contents

1.Preface 2

2.Introduction 4

2.1.Final Problem Statement 4

3.Analysis 6

3.1.Emotional display and understanding in humans – Introduction 7

3.1.1.Emotions and meaning across cultures 7

3.1.2.Culturally Independent Interpretation of Emotions 8

3.1.3.The Facial Expression Program 9

3.1.4.Understand and Forming Facial Expressions 10

3.1.5.The Facial Feedback Hypothesis 12

1.1.1.Analysis Chapter Part 1 Conclusion – Emotions 13

3.2.Emotional Intelligence introduction 14

3.2.1.Emotional Intelligence 14

3.2.2.Emotional Intelligence – A Dual Process Framework 16

3.2.3.Emotional Intelligence – The Four Branch Model 17

3.2.4.Emotional Intelligence – Measurements 18

3.2.5.Emotional Intelligence – Construction Emotions 19

1.1.1.Analysis Chapter Part 2 Conclusion – Emotional Intelligence 20

3.3.Computer Vision Algorithms introduction 21

3.3.1.Automatic Analysis of Facial Expressions 22

3.3.2.Recognising Action Units 23

3.3.3.Smile Detection 24

3.3.4.The Facial Action Coding System 25

1.1.2.Analysis Chapter Part 3 Conclusion – Computer Vision 26

3.4.Affctive Computing introduction 27

3.4.1.Affective Computing in HCI 28

3.4.2.Emotion Expression and Synthesis 29

3.4.3.Affective Computing – Conclusion 30

3.5.Analysis Conclusion 31

3.5.1.Emotions 31

3.5.2.Emotional Intelligence 31

3.5.3.Computer Vision 31

3.5.4.Affective Computing 32

4.Test method 33

4.1.Test Strategy 33

4.1.1.Test Phases 33

4.2.Test Scenario and test location 34

4.2.1.Test Phase One 35

4.2.2.Test Phase Two 35

4.2.3.Demographics 35

4.2.4.Expectations 37

4.2.5.Test Phase One 37

4.2.6.Test Phase Two 37

5.Design & Design Requirements 39

5.1.Design of the website 39

5.2.Computer Software – Smile Assesment 40

6.Requirement Specification 41

6.1.Test Website Requirements 41

6.2.Programming Requirements 41

7.Implementation 42

7.1.Smile assessment Implementation 45

8.Test Results 51

8.1.Test phase one 51

8.1.1.Test Results – Test Subjects 51

8.1.2.Test Results – Computer 53

8.2.Implementing Test Results in the Smile Assessment Algorithm 55

8.3.Test phase two 56

8.3.1.Test Results – Test Subjects 57

8.3.2.Test Results – Computer 58

9.Discussion 60

9.1.Test Method 60

9.2.Open Source Software 60

9.3.Implementation of the Smile Estimation in Phase Two 60

9.4.Errors in the smile estimation 60

10.Conclusion 61

11.Future Perspectives 62

12.Bibliography 64

13.Table of Figures 66

14.Table of Charts 67

15.Appendix 68

15.1.Test Results – Test Phase One – Test Subjects 69

15.2.Test Results – Test Phase One – Computer Results 69

15.3.Test Results – Test Phase Two – Test Subjects 70

15.4.Test Results – Test Phase Two – Computer Results 70

15.5.Source Code – Smile Assessment Implementation 71

15.6.Source Code – Face Detection [DISCARDED] 78

15.7.Source Code - Test Website 88

Download 0.97 Mb.

Share with your friends:
  1   2   3   4   5   6   7   8   9   ...   19

The database is protected by copyright © 2022
send message

    Main page