Face recognition determines moods of bank customers

DECEMBER 31, 2008--Who knows what evil lurks in the hearts of men? Prabir Bhattacharya, Canada Research Chair in Information Systems, or at least his computer, might. For the last three years, Bhattacharya (Concordia Institute for Information Systems Engineering; www.ciise.concordia.ca) and graduate student Abu Sayeed Sohail have been working to develop a computer vision system that detects and classifies human facial expressions. The results of their research to date were recently published by Verlag Dr. Müller in Classification of Human Facial Expression: A Prospective Application of Image Processing and Machine Learning (2008). Why would you want to teach a machine to know if someone were happy or sad? As Bhattacharya explained, Japanese banks have been using computers to judge customer mood for quite some time. Small digital cameras snap photos of each face entering the bank, sending them through an internal network to a computer where they are compared with a database of known facial expressions. By examining details like the lift of the lips and the position of the eyebrows, the computer can make a pretty good guess about whether the customer is happy, tired, angry, or sad. "Tellers are then alerted to customers' moods and can serve the client accordingly," explained Bhattacharya.

Facial expressions do not actually involve the entire face, but rather specific sets of muscles under the face near the eyes, nose, and mouth. Bhattacharya's and Sohail's system measures 15 key points on the face and then compares these measures against images of identifiable facial expressions. Although there is great variety in expression across both individuals and cultures, they have identified seven basic expressions that seem to be relatively universal. And for which their system is fairly accurate.

"We tested using query images against stock databases and the system did quite well, even against the Japanese Female Facial Expression Database (JaFFE)." According to Bhattacharya, Japanese women are the litmus test for all facial expression recognition. "They, and actually Chinese women, generally have less variation in their expressions than most other people on the planet, including Japanese men. If you do well against JaFFE, the program can be applied with good result just about anywhere."

For more information, go to: http://cjournal.concordia.ca/archives/20080925/reading_faces_reading_minds.php