By Devanshi Mehta, PGDM student, and Vineeta Dwivedi, faculty at SP Jain Institute of Management and Research (SPJIMR)
Feeling low? How about a classic Belgian chocolate ice cream, says your food delivery app? Ah, yes Belgium, thinking of a holiday? Here, check out airfares and book your flight on the booking app. Worrying about finances? Here is an instant loan offer from your bank. You get the message!
Understanding consumer behavior has always been paramount for marketers, and in the digital world, brands know how to follow us online through cookies. The next big thing they want is to track your emotions. Emotional Artificial Intelligence (Emotional AI) is here to fulfil your needs by tracking your mood. For marketing professionals, this is the next frontier. AI technologies are advancing fast to learn and analyse human emotions and use that information to improve marketing campaigns.
Meta (Facebook and Instagram’s parent company) has created headsets that can read your mind by tracking your eye movement. They have already developed a system to read your facial expressions and adapt the content. Apple has just announced that a new health coaching service will track emotions using data from Apple Watch and AI.
Brands hope to drive customer behaviour through Machine Learning and Artificial Intelligence. They can leverage changes in facial expressions through computer vision technology, bio-signals, body language parameters, or on commands the customer gives in the voice shopping process. The latter involves several factors like word choice, speech intensity, tone, speech rate, and voice frequency. For example, raising eyebrows, arching into a smile, and using impolite words indicate a positive mood. On the other hand, raising your voice and using many courtesy words may indicate a negative mood.
Emotion AI, also known as affective computing, is a term coined by Rosalind Picard, who did pathbreaking work in this field, which is expected to grow at the rate of CAGR 16.7% in the coming decade. According to a 2023 report by Fortune Business Insights, the market size is projected to reach approximately USD 74.80 billion by 2029. Besides being used by advertisers, it has applications in industries like health, automotive, robotics, insurance, education, and public security. Companies like IBM are also looking to create personalised user experiences by adding more human-like characteristics to Personal Assistant Robots, who are key candidates for this technology.
If the salesperson can know the prospect’s likelihood to purchase before pitching them a product, it could save the company much effort. They could also offer you their product right when you are likely to buy it. Even The New York Times declared that it collected information from its readers over one year to create a list of 30 commonly experienced emotions in various stories using machine learning. “When readers are emotionally aligned with the content, they feel more receptive to the messaging, which increases the yield of positive brand effects,” says Allison Murphy, Senior Vice President of Advertising Innovation at The NY Times.
A 2022 study by Ingo Halbauer and Martin Klarmann shows that retailers can predict the mood at the beginning of the shopping journey using a very small number of commands. This would enable them to gauge user sentiments and adapt the presentation of product information accordingly.
Another remarkable example of brands using technology to build a rapport with consumers would be the video game Nevermind. It collaborated with Affectiva- an AI-based emotion detection software that uses facial and voice recognition to detect complex cognitive states, interactions, and emotions. The difficulty level of the game increases as the player gets more frightened and vice versa.
It is evident from these examples how targeting through Emotion AI allows advertisers to control the context in which the consumers experience the brand. The International Science Council is currently working on producing a series based on a 2018 award-winning study by BBC and StoryWorks that combines emotion-tracking techniques with neuroscience. It uncovered how memorable content is directly linked to the number of emotional peaks the consumer experiences. Companies can, therefore, ‘ride’ memory moments by seamlessly integrating their brand at a key intensity moment. This helps build brand equity and drives brand impact sustained over a long period. The technology can also help reduce post-purchase dissonance, i.e., the uncomfortable feeling a consumer faces after purchasing a product. Real-time emotion analysis and personalised purchase experience make them confident about their ultimate product choice.
Privacy and Accuracy Concerns
While Emotion AI does save time and money by providing quick consumer insights, privacy is a major issue that may arise soon because legal consent is optional if AI does not single out a person. Consumers have no option but to trust the companies to safeguard their data. Moreover, mood targeting can backfire for a brand if it fails to understand the underlying factors giving rise to that mood. It can lead to a mismatch between the resource allocation and emotional states, i.e. ‘misallocation loss’ as termed in a 2022 research. After all, moods are transitory, and if a brand does not target a consistent mood, it might risk losing its identity. This also requires brand strategies to constantly change along with the consumer’s moods.
Humans are also very proficient at hiding emotions, on account of which the technology would give inaccurate and biased results at times. Experts say that AI technology is based on a ‘flawed science’ as it does not consider the context and situation a person is in while analysing emotions. Researchers are therefore looking at models to combine Natural Language Processing (NLP) with nonverbal signal computations to give accurate results. High-quality data, cultural differences, and computational load are some primary challenges. Having information that is timely, relevant, and actionable is also extremely important for advertisers.
Then there is the ethical question if this technology is used for sensitive issues such as determining an employability score for job candidates, proctoring system in examinations, visa and immigration, etc. It may result in unintentional bias, thereby negatively impacting individuals, as emphasised in a 2022 study that says we need to be wary of emotional AI.
Treating people’s emotions as ‘just another metric’ through which they can be mapped may even seem inhumane to some. However, given the remarkable opportunity for companies to create new value, the march of the machines is on! The irony is that while these technologies aim to bridge the gap between humans and machines, they might end up doing the exact opposite.