By Khalida Sarwari
We see the adage plastered everywhere: “Don’t judge a book by its cover.” And yet, we do it every day.
We judge people’s facial expressions in social interactions, of course, but we also do it as jurors in the courtroom when deciding whether a defendant is guilty or remorseful. Assumptions about people’s feelings based on their facial movements influences treatment plans for people living with autism and other brain disorders, and it has shaped the direction of research in various scientific fields, including neuroscience and psychiatry.
It turns out, there’s something to that adage. Gauging a person’s emotional state from their facial expressions alone is a futile exercise, says Lisa Feldman Barrett, a University Distinguished Professor of Psychology at Northeastern who studies the way humans express emotion.
“The fact that people miss-guess what facial movements means is not new,” she says. “What’s new here is that we are showing that the evidence never suggested that facial expressions are universal despite the claims that are being made by some scientists and by many companies.”
Barrett and a blue-ribbon panel of five senior scientists reached this conclusion after spending two years reviewing hundreds of experiments and existing studies to test the commonly held belief that facial expressions are universal.
“In our case, the question of whether there are universal facial expressions—whether you can read someone’s emotion in their face—is a pressing issue in national security, in health, in industry, in education, and actually in a variety of domains,” Barrett says.
The researchers examined the facial expressions of adults living in urban western cultures, as well as children and infants, congenitally blind individuals, and people living in remote cultures. The results of their research suggest that how people convey anger, disgust, fear, happiness, sadness, and surprise varies considerably across cultures, situations, and even across people experiencing the same incident. Barrett adds that some facial movements, such as a scowl, often communicates something other than an emotional state. People also scowl when they’re confused or concentrating on something, she says.
Our misguided assumptions about the correlation between facial expressions and emotions have significant implications for various sectors of society. Barrett cited a behavioral screening program by the U.S. Transportation Security Administration that trained agents to spot terrorists by reading facial expressions that has been deemed ineffective.
“Basically, the government spent almost a billion dollars on a training program to keep citizens safe, [but] the evidence in the literature was there all along to show that it wouldn’t work,” Barrett says.
She also cited software programs purporting to read emotions in faces that are being deployed or tested for a variety of purposes including surveillance, hiring, clinical diagnosis, and market research. Microsoft, for example, has an algorithm that claims to detect what an individual is feeling based on video images of their face.
“What those companies can read under ideal conditions is whether a face is conforming to a scowl or not, but not what the meaning of that is,” she says. “It’s important to know how to interpret what you’re reading.”
All this to say that facial expressions aren’t completely devoid of meaning, Barrett says. After all, people in western cultures by and large do scowl when they’re angry or smile when they’re happy. But in order to really understand how someone is feeling, it’s important to consider the context around their facial movements, and also take into account variables such as the individual’s physical state or body posture and vocal acoustics.
“It’s clear that facial movement conveys meaning much of the time and we now have the tools and the technology available to understand what facial movements mean in all of their glorious variety if we just start asking the right questions and designing our experiments differently,” says Barrett.
