An analytical framework for studying attitude towards emotional AI: The three-pronged approach

Introduction: The rise of emotional AI and a new era of human-machine relation

Emerging emotion-recognition technologies are grounded in the pioneering work of MIT Media Lab's Rosalind Picard, who coined and defined the term ‘affective computing’ as “computing that relates to, arises from, and deliberately influences emotion” ([3], p. 13). The commercial moniker ‘emotional AI’ has come to define a growing multidisciplinary field that draws on computer sciences, engineering, psychology, physiology, philosophy, and neuroscience to enable algorithms to sense, track, classify, and respond to human emotions and affective states [21].

AI research is broadly recognized as two distinct paradigms, Weak and Narrow AI (AI focused on specific automated tasks and carrying rigid capabilities) and Strong AI or Artificial General Intelligence (AI that performs learning and thinking on varied tasks like a human). A good example of Weak AI is in John Searle's (1980) famous Chinese Room thought experiment (Wakefield, 2003) where he suggested a machine that can only appear to have intelligence without having any real understanding of the world and its action. While no examples of Strong AI yet exist, leading AI thinkers such as Hubert Dreyfus (1972), Margaret A. Boden (1990), Marvin Minsky [24], and Nick Bostrom [2] have debated whether General AI is possible. Many experts believe that sentient machines will require unimaginable breakthroughs in theoretical and technical knowledge. Still, popular imagination is inundated with representations of Strong AI (Astro Boy, Terminator, Ex Machina, etc.), artificially intelligent agents that think, feel, and act like humans.

Current emotional AI systems are a weak and narrow form of AI, they use an ensemble of methods from machine learning, natural language processing, and knowledge-based approaches to teach machines to read, categorize, and react to human emotions. They also harness biosensors, cameras, and actuators to monitor and harvest biometric data from an individual's affective state. This includes heart/pulse/respiration rate, skin perspiration levels, head and body movements as well as facial micro-expressions. While emotion-sensing technology can identify and evaluate the psycho-physical details of an individual, it has no subjective awareness of the emotions it is trained to recognize. For example, the machine might correctly identify anger but has no self-awareness or subjective experience of anger itself, and its ability is limited to only what is programmed. Fig. 1 presents an emotional AI product, co-developed by Nippon Electric Corporation (NEC) and RealEyes, a UK start-up [26]. This AI system tracks in real-time emotional states of video conference participants, e.g., attention, relaxation, and confusion, and produces visualizations of their changes during different stages of a conference.

Applications of emotional AI

According to Kate Crawford [6], the emotional AI industry is worth around $22 billion and is expected to double by 2024. Applications for the technology are rapidly expanding with each passing year. For example, the music app, Spotify is known for its ability to create personalized playlists based on algorithms that look at what a user has played, saved, liked, and shared has now patented a new feature that analyses a user's voice and then recommends songs based on their ‘emotional state, gender, age, or accent’ [14]. NEC, a Japanese security conglomerate, has developed software for McDonald's that measures customer sentiments as they are looking at digital menus to optimize the customer's experience while at the same time increasing sales [31]. Similarly, AdMobilize and Affectiva market AI software linked to public security cameras that monitor audience responses to ads. Besides analyzing gender, age, and engagement time, these companies use facial analysis to detect micro-expressions of happiness, surprise, neutrality, and dissatisfaction. In the automotive industry, legacy companies such as BMW, Audi, and Honda are upgrading their ‘in-cabin’ experience with sensors and actuators that detect a driver's emotional states, e.g., happiness, anger, alertness, distraction, calmness, or agitation, etc. The application's is designed to provide customers with personalized driving experiences and health alerts [5]. On the domestic front, the Japanese company Gatebox markets Azuma Akiri, a smart hologram in the image of a ‘cute girl character’ (bishojo-美少女). Intended as a home assistant for male otakus (anime/manga lovers), Gatebox's hologram is pushing the boundaries of human-machine relations. In terms of security, global market player ELSYS markets video analytic software which they claim can detect a wide array of mental states and suspicious behaviors (www.elsys.com). Elsys's malicious actor recognition technology can be found in casino, airports, nuclear power plants, ATMs, commercial and retail outlets, private security companies and government police forces across the globe [38]. While in education, emotional AI vendors are touting their wares as a preeminent technological fix for distance learning. From Chicago to Hong Kong, schools are using emotion recognition software to monitor children's emotions (happiness, sadness, anger, disgust, surprise, and fear) in the classrooms to gauge ‘motivation’ and even forecast grades [6].

With the rise of smart cities, Internet of Things (IoT), and ubiquitous computing, emotional AI will become an integral layer of daily life. Critically, emotional AI is not merely a passive observer or recorder of human emotions. Opponents of the technology warn of its negative potential, how it may lead to not only unwanted emotional policing and attitudinal conformity in institutional settings but also to the polarization of public discourse [19]. In the wrong hands, emotional AI may be a threat to human privacy and autonomy [17,21,37]. Thus, while emotional AI can be a tool for social good, it may also do serious harm. How do we live well and ethically with machines that feel and feed off our emotions? What is needed to ensure that affect recognition technology serves the best interests of society? What are the benefits gained from interacting with machines that can sense our emotions? What are the risks involved? What does co-existing with emotional AI mean for values that we cherish: privacy, autonomy, fairness, trust, etc.? How do we begin to parameterize the ethics of emotional AI?

Answering these questions depends on making sense of our perceptions of the technology and its impacts on our life. This article proposes a method of quantitatively studying social and ethical perceptions of emotional AI. We argue there are two intuitions at play in determining the acceptance of smart affect-sensing technologies.

Our first intuition is the degree of acceptance of technology increases with its utilities: its ease of use, its help in managing stress, improving security, and improving overall well-being. This intuition is captured and formalized in the Technological Acceptance Model [7], which postulates perceived utilities and perceived ease of use as two fundamental factors in technological acceptance. The second intuition is acceptance of a new technology is contingent on a person's perception of its risks: its capacity for intrusiveness, or potential to diminish a user's sense of autonomy and freedom of expression as well as undermine social/public trust, etc. The latter is captured and formalized in the Moral Space, which Hidalgo et al. [12] built upon the Moral Foundation Theory by Haidt [11]. Hidalgo et al. [12] propose how humans judge AI is a function of how the machine violates or conforms to the five moral dimensions: Harm, Fairness, Loyalty, Authority, and Purity.

Critiques of the technological acceptance model (TAM)

TAM was first proposed by Fred Davis [7] and remains one of the most well-cited models in the study of tech-adoption behaviors. In the original TAM model (1989), Davis hypothesized that the acceptance of new technology is determined by two major factors: perceived ease of use and perceived utility (See Fig. 1). In a later paper published in 2000, Davis's model was extended to include subjective norms, which he narrowly defines as whether most people who are close to (or familiar with) a person think he or she should or should not adopt a technology (p.187). In other words, it presumes a measurement of conformity due to social influence. Both the original (1989) and the extended TAM [34] have enjoyed a high level of citation and empirical support. For example, a study found the extended model accounted for 61% of the variance in the behavioral intention (BI) to adopt mobile wallet technology [16]. Other meta-analyses of digital technology adoption in education show the TAM models can account for up to 44% of the variance in the BI [30].

However, the rise of emotional AI and other forms of smart technologies such as ubiquitous computing or IoT is embedded in our physical and digital infrastructures and thus, operate in a nescient fashion. AI users may not be users in the traditional sense as the technology is often designed to run in the background without inciting awareness or consent [15]. This is especially true of affect recognition devices that harvest non-conscious biometric data. As such, emotional AI along with other smart technologies exposes several limitations of the TAM, namely, its linear, static, and tactile assumptions of human-machine relationships as well as its cultural insensitivity.

First, TAM implies a linear subject-object relationship between a user and a technology. Indeed, there is an act of the user physically (or often consciously) adopting the technology. Such a linear and tactile relationship is no longer a prerequisite with emerging smart technologies and coded spaces (Kitchin & Dodge, 2011), which often operate in a ubiquitous, ambient fashion in the background of personal devices or public spaces.

Second, affect-sensing algorithms are not static. Rather, besides being able to read and track our emotions, they can also respond to our various emotional states, and as demonstrated in the literature, they are often designed with transparent but also ulterior aims. For example, affect tools may be used in mental health counseling [4,10], or in optimizing workplace productivity [18]. Conversely, they can be used as a tool of behavioral control, maximizing certain desired behaviors [37,39], including purchases of and engagement with contents and products on online platforms as well as a disciplinary tool, ensuring students pay attention in class. Thus, as AI-powered physical and digital platforms direct and nudge our behavior and attention, this new human-machine relationship dictates novel ways of conceptualizing models of technological acceptance.

Finally, there is also a crucial issue of the TAM not being culturally sensitive. A common critique that has been leveled at TAM is its lack of accountability for the cross-cultural variance (differences in core values, mindsets, etc.) in the way people form acceptance perceptions such as ease of use, utility, and social influences [33]. Indeed, emerging literature that tests the TAM and TAM2 models in various countries shows cultural values do indeed influence how people form the perceptions that are pertinent to the TAM [8,20,25]. Hence, to fully understand the perception of emotional AI technologies, the TAM model must be supplemented by other theoretical frameworks.

The moral space: Applying the moral foundation theory to study the human-machine relationship

The Moral Space is a mathematical construct that Hidalgo et al. [12] used to explain, quantitatively, the perceived morality of a machine's actions. Its origins stem from Jonathan Haidt's Moral Foundation Theory [11], which posits five fundamental moral dimensions including fairness, loyalty, harm, purity, and authority (Fig. 2). In How humans judge machines (2021), Hidalgo and colleagues propose the morality of a machine's action can be captured by a function of how it has violated or validated the five moral norms laid out in Haidt's Moral Foundation Theory.

The Moral Foundation Theory proposes five moral foundations. The first is the dimension of Harm/Care, which is the concern about and dislike for the suffering of others. The second is the dimension of Fairness, i.e., the concern for proportional versus egalitarian fairness (Fairness). The third is Loyalty, i.e., the concern for ingroup loyalty (Loyalty). The fourth is Authority, i.e., the concern for preserving social structures, authority, and tradition. The last is Purity, the concern prompted by the feeling of disgust about physical or mental/spiritual contamination. The first two foundations, Harm/Care and Fairness are often regarded as individualizing foundations since they entail the concern for the well-being of a person. The latter three foundations are considered the ‘binding foundation’ since they are about maintaining cohesion and order in the collective [1].

In their work, Hidalgo et al. presented hypothetical, but not far-fetched scenarios to nearly 6000 subjects involving machines or humans making consequential decisions in different contexts. For example, a machine doing job screening vs a human doing job screening, a machine vs human security guard determining the legal status of immigrants in an airport, etc. The authors, using a 7-point Likert scale, ask the respondents in the United States to rate the moral wrongness of such situations. They proceeded to ask the respondents to rate how much a given action of a machine and a human has violated a moral norm in the Moral Foundation Theory.

This approach is quite suitable for social scientific research of emotional AI because it enables researchers to interrogate various ethical dimensions and concerns related to the technology, e.g., as a threat to privacy, as a threat to autonomy, or to examine its utility such as increased safety or intimacy. It also highlights the contingency of human evaluation toward machines. The researchers found that people tend to judge machines more harshly based on the outcomes rather than intention . Moreover, uses of AI by state-actors are judged differently than uses of AI by their non-state counterparts.

Contexts

Regarding the importance of context, there are two operational definitions of the word ‘context’. First is the context of affect-recognition technologies use, for example, workplace, healthcare, home, security, education, car, etc. Ample evidence suggests each different use case will bring about different sets of concerns or make certain concerns more pronounced than in other cases [9,13,18]. A good illustration is in the automotive industry, where McStay and Urquhart [23] have pointed out that there are currently no collective industry standards for emotion-sensing devices now deployed as a propriety layer in many new car models. Similarly, in the emotional AI toy industry, McStay and Rosner [22] argue that there is a ‘generational unfairness,’ in which, children, unlike adults, have little control and ability to negotiate the uses of these technologies. Another crucial and relevant example is the workplace, where tensions related to automated management systems are impacting workers’ rights to privacy, autonomy, and inclusivity [18].

Second is the cultural context, i.e., how cultural elements play an important role in shaping our social and ethical perceptions of smart technologies. Culture influences perceptions of the risks and rewards of technology. For example, in communal contexts such as the workplace or healthcare settings, Westerners are more averse to data collection than Asians due to the latter's trust in institutions and authority. Previous studies found to regard the data collected in these settings to be less sensitive, thus professing a higher level of acceptance compared to the Western subjects [18,28]. The same follows for how traditional values and norms in parenting, employer-employee relation, education, etc. vary across different cultures.

Thus, in designing a social scientific study that involves emotional AI products, there is a clear need to identify the context of the technology's usage as well as the cultural setting or environment in which usage takes place. Rather than applying a generic theory and adding new variables in a seemingly arbitrary manner, clearly identifying context will allow researchers to identify important variables in modeling the acceptance of emotional AI technologies. Importantly, understanding the implications of context allows the researchers to bring clarity to the wording when they construct the items, i.e., how questions and statements in a questionnaire are phrased to appropriately reflect nuances in the culture and context of technology use. A good example here is HireVue, the US hiring platform company, recruits native experts to translate their interview questions for job candidates entering the Japanese market in order to make them feel less ‘direct’ and ‘impolite’ to the Japanese ears.

Variables

Once the context of use and cultural nuance are identified, the TAM and the Moral Foundation Theory can provide the initial variables to better examine the social and ethical perceptions of smart technologies.

First, based on the TAM, utilities such as ease of use, knowability, perceived increased safety or productivity, etc. play a determinant role in attitude towards the technology. Second, based on the Moral Space, harms conceived as a violation of moral values such as privacy, fairness/inclusiveness, and autonomy will also influence our acceptance of the technology. Yet, with each context, the wording as well as the number of survey items for each variable can vary slightly. Table 1 provides an example of how to design a survey to understand users' perception of emotoinal AI according to the three-pronged approach proposed in this paper.

Here, it should be noted this survey method can encounter various obstacles such as survey fatigue or the social desirability bias, ie., when the respondents answer in ways that they believe look good to others, or the acquiescence bias, i.e., the tendency of the respondents to agree with survey statements rather than stating their true intention. For the issue of survey fatigue, it is advisable that researchers control the volume of research items and limit the number of survey contexts (i.e., education, security, politics, media, healthcare, etc.) to a minimum to mitigate this issue. Regarding the social desirability bias and the acquiescence bias, it is advisable that an online, anonymous survey be used to minimize the in-person contact that might lead to these biases.

Statistical models

Once the variables are identified, in constructing the survey an effort should be made to scan for any wording that may be construed as ‘culturally in/sensitive’. Afterward, in the modeling process, assimilated causal diagrams as popularized and extensively applied by Pearl and Mackenzie [27] are encouraged. The reason for this is twofold. First, causal diagrams make transparent the assumptions involved, forcing the researchers to be precise in their inferences. Second, such transparency also aids researchers in making more precise and deliberate choices in the statistical tools used to analyze the survey responses.

Below are some causal diagrams that seek to explain/gage behavioral intention to adopt emotional AI technology. Indeed, the causal diagrams make explicit the hypotheses regarding different relationships among the studied variables. For example, in Fig. 4 below, Model 1 presumes variables from the TAM and from the Moral Foundation Theory influence the attitude toward the technology in a linear, non-hierarchical way - the perceived concerns and utilities all mediate the attitude toward the technology, which in turn mediates the behavioral intention to adopt it. Here, if a researcher only wishes to investigate determinants of attitude toward the technology, i.e., Model 1 minus the behavioral intention to adopt, he or she might use the Ordinary Least Squares (OLS) method. The OLS method is used widely by quantitative social scientists for studying linear relationships among variables while also accounting for variance in individual characteristics (such as by sex, age, educational qualification, income, etc.). It is also used by Hidalgo et al. [12] for analyzing how moral concerns influence our judgment toward machines. Meanwhile, Model 2, the effects of perceived utilities, ease of use, and self-rated knowledge regarding the technology are hypothesized to be filtered through, i.e., modulated by, an individual's core moral concerns (a prediction made by the mindsponge model of acculturation [35,36]. In this case, structural equation modeling (SEM) is a viable option for conducting the statistical analysis on Model 2.

Besides the suggested OLS and SEM models, other advanced statistical methods are also recommended for the types of problems posed in this study. We believe the proposed Three-pronged approach yields well to advanced statistical methods such as Bayesian Hierarchical Analysis. However, the followings are some key considerations to think about applying these advanced methods. Based on the Three-pronged approach, background socio-demographic factors such as sex, gender identity, ethnicities, religions, regions, income level, educational level, etc. should be collected. Knowing the distribution based on these factors allows researchers to make precise choices regarding what statistical analysis methods to apply. For example, when there is an unequal representation of some demographics, Bayesian Hierarchical Modeling has been shown to be better than the traditional frequentist method to tackle the unequal representation [32].