06.1.2022 Amy Dampier, VP Customer Success
Sure, in today's world, retailers and brands are swimming in a sea of data. But some information remains elusive—namely, consumer emotion. Save for direct feedback or the occasional survey, the industry knows very little about how shoppers feel about the products on the shelves or the displays customers see before they make a purchase (or not).
A subset of artificial intelligence (AI) technology may hold the key to collecting this type of information in the future for bricks-and-mortar. Known as emotion AI, this technology relies on software to classify captured facial expressions into categories that include joy, sadness and surprise. Emotion AI can offer a read on consumers' feeling throughout the in-store experience, and companies are beginning to invest in and develop these solutions in order to turn shopper sentiments into purchasing actions.
Capitalizing on Shopper Sentiment
Sentiment is a powerful measurement tool. How people feel—how confident, happy or disappointed they are—could go a long way toward suggesting how they will act. In the investing community alone, sentiment has been the subject of a wide range of study, and consumer confidence and sentiment indexes, such as the University of Michigan's Consumer Sentiment Index, hold great importance as predictors of consumer behavior.
In retail, sentiment works in much the same way: How shoppers feel about a given product, brand or retailer is directly correlated to the decisions they make about a purchase. This is why emotion AI holds so much promise for retailers and brands—it can identify this sentiment. As Mario Noioso, digital architect at Microsoft, recently wrote in his Internet of Ideas blog, “[...] if a consumer walks up to a display and looks frustrated, retailers will know that something needs to be tweaked. Joy would prod the retailer to buy more of that good. Stores could send help and personnel to folks depending on their emotional response to an item."
Noioso also suggested another retail application: A camera connected to an emotion AI analysis tool could be placed behind a shop window to record real consumer feedback in the form of facial expressions about the shop window set-up, helping retailers and brands make decisions about what to put in their window displays.
While new ideas like Noioso's continue to emerge, the application of emotion AI to retail is still in its infancy. One company leading the way is retail technology company Cloverleaf which equipped its ShelfPoint system with emotion AI software created by Affectiva. ShelfPoint is a system of digital shelves that uses LCD strips to display marketing messages to catch shoppers' attention from afar. Then, once shoppers come within a few feet of the LCD strips, optical sensors collect data about consumers' expressions and the emotion AI software analyzes it for sentiment. When used in a shopping simulation test at Proctor & Gamble's Beckett Ridge Innovation Center, these displays showed an ability to potentially deliver double-digit percentage sales increases when compared to promotional displays without the technology.
A New Advantage for Brick-and-Mortar Over Ecommerce?
Brands and retailers are at a pivotal moment with brick-and-mortar. As titans and long-standing leaders of the industry fail to evolve and lose out to newer and more innovative brands and retailers, those who remain search for new avenues to connect to and understand consumers in ways that their ecommerce counterparts cannot. Emotion AI is one way bricks-and-mortar can continue to compete.
In order for emotion AI to help brick-and-mortar maintain the advantage over ecommerce, though, it needs to be used in the ongoing effort to achieve a complete picture of the shopping experience. Cloverleaf, for example, hopes to integrate even more data collection and analytics capabilities into ShelfPoint, with features now in development that could recognize when a shopper takes a product off a shelf, how long it remains in their possession and when, or if, the product is placed back on the shelf.
“As a retailer or brand, you're trying to capture, control and convert," Gordon Davidson, founder and CEO of Cloverleaf, says. Capture the shopper’s attention with visually stimulating displays, control—or influence—their behavior with additional messaging, and convert them from browsers into buyers.
Emotion AI can give retailers a sense of how close they are to that trifecta, and as companies like Cloverleaf, Microsoft, Apple and others continue to adapt emotion AI solutions for retail use, bricks-and-mortar retailers should take a closer look at these solutions.
