Wearable Device Tracks Consumer Emotions to Products and Experiences

Humans experience a variety of emotions in response to experiences and products everyday. Shoppers may get enthusiastic for some brands and then overwhelmed by the wide selections. Audience members may waver between engagement and apathy during performances. Children can become entertained, frustrated, or bored, while learning a new subject.

MIT Media Lab spinout mPath is able to identify the precise moment consumers experience these subconscious responses by using wearable stress sensors, analytics and other technologies. In doing so, the startup has provided some remarkable market research acumens to major organizations and companies to help them upgrade their products and services.

Right now, companies struggle to understand their customers’ emotional needs or wants. But if we listen a little to consumer emotions, there’s a lot of room for innovation.

Elliot Hedman PhD ’12, Founder and CEO

mPath, which has its Headquarters in Denver, Colorado, has worked with clients such as Google, The LEGO Group, The Blue Man Group, Hasbro, Lowe’s and Best Buy, as well as film production companies, governmental organizations and hospitals. Presently, mPath is entering into childhood education, having recently launched a project with the Boys and Girls Clubs in Denver to find ways to inspire children to read more.

Getting the whole story with “emototyping”

The startup’s MOXO sensor — whose main technology was co-invented by Hedman, MIT Professor Rosalind Picard, and other MIT Researchers — is a wearable that looks like a bulky smartwatch. Positioned on the wrist, it wirelessly measures alterations in skin conductance (mild electrical changes across the skin), which reflect physiological arousal and sympathetic nervous system activity. Spikes in conductance can indicate frustration and stress, while dips may signal boredom or disinterest.

To get an accurate idea of consumers’ responses to particular stimuli, mPath formulated a new approach to market research, called “emototyping.” This process integrates the stress sensors with GoPro cameras or eye-tracking glasses, to identify what a person viewed at the exact moment of an emotional dip or spike. Personal interviews are also conducted with the participants, who were then shown the data and asked what they think they sensed.

This whole process develops a more in-depth, precise emotional profile of consumers than traditional market research, which mainly involves interviews and sometimes video analysis, according to Hedman. “All these things combined together in emototyping tell us a deep story about the participant,” he says.

Emototyping is a particularly useful tool when examining children’s experiences, according to Hedman, “It’s hard for kids to describe what they felt,” he says. “The sensors help tell the whole story.”

Insights from tooth-brushing, to retail, to reading

In the past, the startup has undertaken a diverse mix of projects. A research with the New World Symphony found that composing shorter songs and performing classical compositions of contemporary pop music help engage new audiences in classical music. Analyzing movies such as “The Departed” exposed where some methods or concepts (such as dark humor) can be implemented in films to sustain the interest of audiences. In one particular case, the startup even tracked patrons’ fear while moving through parts of a haunted house.

One of mPath’s more unique latest projects was helping a toothpaste company get an insight into people’s experience while they brushed their teeth. “I’ve learned so many things I never thought I’d know about brushing,” Hedman says, laughing.

For the research, mPath Researchers went to people’s homes to fit them with the MOXO sensor and a GoPro camera. At times, they measured few to no skin-conductance spikes in the data, signaling the brushers were bored; at other times, little spikes in the data revealed the brushers were disturbed. After analysis and interviews, mPath determined that people are seeking a more engaging brushing experience. The toothpaste company is designing products around that idea.

More frequent is retail-store research. In analyzing customers in electronics stores, mPath found that engagement spiked while they operated interactive electronics, but dipped considerably when an employee came to rattle off a “sales pitch.” Results like this specify that retail stores must better equip employees to interact with customers and create displays that better inspire deep interaction, “which is a big space where retail needs to develop,” Hedman says.

Presently at the Boys and Girls Club, mPath has discovered techniques to motivate reading. Hedman provides an example of a young girl who seemed to be reading a book, but the eye-tracking glasses tracked her focus, which shifted to reading a poster on the wall.

Our theory is that an entire paragraph can look overwhelming, so we’re developed some ways to make text look more like a poster — something easy to read if they’re trying to learn.

Elliot Hedman PhD ’12, Founder and CEO

In another case, mPath countered a generally held idea that allowing children to play a short video game as “dessert” after reading works well. But mPath’s data discovered that when children play a short game before reading, the sensors pick up more spikes in engagement. “We found if children start with dessert, they’re more excited during reading,” Hedman says.

From therapy to industry

The MOXO sensor’s key technology started as a tool for examining stress levels of children with autism. (A version was later built into the E4 wristband that can sense future seizures, marketed through Picard’s startup Empatica.)

While at MIT, Hedman analyzed the sensor on children with autism registered in occupational therapy. One boy was climbing a rock wall and seemed calm, but his stress spikes were very high. “I thought, ‘The sensor has to be broken. There’s no way that kid is scared,’” Hedman says.

After the climbing excercise, the boy told the Teacher he was bored, but the sensor data showed otherwise. That is when the Teacher had, “a light-bulb moment,” Hedman said the boy said he was bored to make up for being overwhelmed.

That was a really powerful moment for me. I knew there is so much we can do if we understand emotions better.

Elliot Hedman PhD ’12, Founder and CEO

Therapy applications looked promising. But companies started seeking Hedman out with interesting market-research questions. Hasbro was the first. They wanted to learn how immigrants to the United States can understand how to play popular American board games, such as Monopoly. The next company was LEGO, which wanted to track the emotions of adults and children at play — which made Hedman move toward industry.

In the LEGO study, Hedman discovered that children became most stressed when making errors they need to retreat a few steps to correct. It also showed the children were far less stressed with parents in close proximity. But if parents assisted them to build too much, the children got too bored. Based on those results, LEGO executed solutions into its digital products.

“I was like, ‘Cool, we just helped a company.’ That kind of shifted me from theory to impacting organizations,” Hedman says.

Hedman founded mPath in 2012, while at MIT. But the sensors were only part of the innovation. Hedman needed to create “a bizarre set of skills,” he says, to develop emototyping. At MIT, he signed up for classes on ethnography and developing research environments. He also took psychophysiology classes at Boston College and Northwestern University, and education classes at Harvard University. “A large community of people from all over the place guided me,” he says, “but MIT opened up those doors.”

In the days to come, mPath will pursue doing “impactful work,” Hedman says, such as exploring young student engagement at physical schools and in massive open online courses (MOOCs), with aims of helping redesign classroom experiences and curriculum.