Skip to content Skip to navigation Skip to collection information

OpenStax_CNX

You are here: Home » Content » ECE 301 Projects Fall 2003 » Approach

Navigation

Table of Contents

Lenses

What is a lens?

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

This content is ...

Affiliated with (What does "Affiliated with" mean?)

This content is either by members of the organizations listed or about topics related to the organizations listed. Click each link to see a list of all content affiliated with the organization.
  • Rice University ELEC 301 Projects

    This collection is included inLens: Rice University ELEC 301 Project Lens
    By: Rice University ELEC 301

    Click the "Rice University ELEC 301 Projects" link to see all content affiliated with them.

  • Rice Digital Scholarship

    This collection is included in aLens by: Digital Scholarship at Rice University

    Click the "Rice Digital Scholarship" link to see all content affiliated with them.

Also in these lenses

  • Lens for Engineering

    This module and collection are included inLens: Lens for Engineering
    By: Sidney Burrus

    Click the "Lens for Engineering" link to see all content selected in this lens.

  • EW Public Lens display tagshide tags

    This collection is included inLens: Ed Woodward's Public Lens
    By: Ed Woodward

    Comments:

    "assafdf"

    Click the "EW Public Lens" link to see all content selected in this lens.

    Click the tag icon tag icon to display tags associated with this content.

Recently Viewed

This feature requires Javascript to be enabled.

Tags

(What is a tag?)

These tags come from the endorsement, affiliation, and other lenses that include this content.
 

Approach

Module by: Sivakiran Nagisetty. E-mail the author

Summary: Solving the problem of mood detection

Control Flow

The classification is based upon a branching flow, with an appropriate test at each node. The emotions are divided into broad categories and further tests are performed to identity each emotion individually. For this, we divided our set of four emotions into good and bad emotions with happy and surprised being the happy moods and sad and angry being the bad moods.

Before any of the tests were performed, the image of the face woudl be cropped to the mouth which is our primary interest. This can be done in one of two ways, either using Matlab's ginput command or the function goodcrop.The highest level test, Test 1, distinguishes between pictures representing a happy or positive emotion from those representing a bad or negative emotion. Once a mood was detected as either being a happy or a sad mood it would then be tested again to classify it as one of the two moods in each category. The two pictures which are identified as having positive emotions are then sent to Test 2 which distinguishes between an happy and a surprised emotion. The other two pictures are sent to Test 3 which distinguishes between sad and angry.

Figure 1: Control flow
 (flowchart.jpg)

This method was quite effective as it reduced the total number of tests to be performed to three. Having a single test to correctly detect a emotion was not feasible because it involved setting thresholds which wouldnt hold over a wide variety of test cases and also increase the total number of tests being performed.

GoodCrop : Image Cropping Routine

This function is used to crop the mouth from an image. It is called with the original image and returns the part of the image containing the mouth. It is based on the sobel edge detector. Due to the image specifications, it is safe to assume that the mouth is located in the lower third of the picture. The function makes this basic assumption and does an inital crop to this size. It then perform the edge detection and counts the number of edge points along each row and column. It also does a 11 point smoothing on the row and column summations to negate the effects of the nose being part of the intial crop.

Figure 2
Figure 2 (crop1.jpg)

It then chooses the peak correponding to the mouth and the extent of the peak to decide on the location and extent of the mouth vertically and horizontally.

Figure 3: Pre and Post Cropping
Figure 3 (crop2.jpg)

Test 1: Good Mood or Sad Mood

This test is used to distinguish between a good mood and a sad mood. It is based on the use of the 2D Gabor wavelet transformation. Use of the 2D Gabor wavelet representation was pioneered by Daugman in the 1980s. A complex-valued 2D Gabor function is a plane wave restricted by a Gaussian envelope.

ψkx=k2x2e(k2x2)2σ2(eikxeσ22) ψ k x k 2 x 2 k 2 x 2 2 σ 2 k x σ 2 2

The Gabor wavelet representation allows description of spatial frequency structure in the image while preserving information about spatial relations. The complex amplitude of the transforms is used to test for features.

Figure 4: Gabor representation of the four moods for subject 1.
 (output1.jpg)

Since the Gabar wavelet highlights and extracts features from a facial image, those images with stronger expressions have greater intensity in their transforms relative to those with weaker expressions. As can be seen from Figure 1 the happy and suprised pictures which are on top have relatively higher values when compared to the sad and angry picctures. After taking the 2-norm of each transform, we are able to classify the strongest two as happy or surprised and the weaker two as sad or angry.

Test 2 : Happy or Surprised

Intuitively, the distinguishing feature between a smile and a surprise is the teeth and a measure of how open the mouth is. While more teeth are visible during a smile, a surprise d emoution has a more open mouth. Because teeth are white (hopefully!) and because white corresponds to a high intensity, we apply a process to determine how much of the mouth is high intensity and consequently, whether the person is smiling or not.

Figure 5: Surf plots for happy and surprised
Figure 5 (output2.jpg)

The detector first creates high-contrast versions of our original cropped pictures. It then designates a threshold value that is 95% of the maximum value, which is usually white, and determines the number of such peaks that exist. The image with the greater number is designated "happy", and the other as "surprised".

Test 3 : Sad or Angry

From the start of our project we had anticipated difficulty in deciphering a sad from angry image since both emotions usually entail a closed mouth. With a closed mouth there was no way for us to compare intensity or even edges, as teeth create in a happy picture. To our advantage early in the project we had consulted with CAAM Professor Dr. Mark Embree, who suggested that we try using a Singular Value Decomposition (SVD) transform to analyze our images.

The equation of SVD:

svdeqn.jpg

Where u.v* creates a m*n matrix that is essentially a basis for the matrix A and σσ is the coefficient relating the matrix A to the corresponding basis matrix. The most remarkable aspect of SVD is that the first few σσ values and corresponding basis are nearly sufficient to reconstruct an image. For our project the matrix A corresponds to the cropped rectangle of the mouth. To utilize the SVD transform we used MATLABS built in function S=SVD(S) where S corresponds to the vector of the values pertaining to a particular matrix. After A few test cases we quickly realized that the norm of S for a sad image compared to an angry image where roughly greater in magnitude by 1,000-2000, or 10%-20%. This discovery allowed us to take the images that have been passed into this branch of testing and correctly discern the sad image from the angry image by a simple comparison. Here is an image to help visualize the method used for this branch of testing:

Figure 6: Image and corresponding svd plot for sad and angry emotions
Figure 6 (output3.jpg)

Collection Navigation

Content actions

Download:

Collection as:

PDF | EPUB (?)

What is an EPUB file?

EPUB is an electronic book format that can be read on a variety of mobile devices.

Downloading to a reading device

For detailed instructions on how to download this content's EPUB to your specific device, click the "(?)" link.

| More downloads ...

Module as:

PDF | EPUB (?)

What is an EPUB file?

EPUB is an electronic book format that can be read on a variety of mobile devices.

Downloading to a reading device

For detailed instructions on how to download this content's EPUB to your specific device, click the "(?)" link.

| More downloads ...

Add:

Collection to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks

Module to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks