Emotion Research:
Cognitive Science / Artificial Intelligence
Computational Models of Affect
Although a number of classification dimensions exist (see Framework
section of this homepage), we have selected the level of abstraction
of the model as the primary organizing dimension for the different models.
We use the following three levels:
A number of models exist which attempt to identify emotions from facial
expressions or other behavioral or physiological measures. These models
do not fit into the categorization above and are discussed separately at
the end of this section.
An excellent earlier review of computational models of emotion can also
be found in Pfeifer, 1988.
Architecture-Level
Models
Models in this category exist at output model end of
the spectrum and at the process model end.
- Bates, Elliot, Reilly.
(see also OZ
Project, CMU)
Model of emotion as part of a larger project whose goal is to construct
autonomous agents inhabiting virtual reality microworlds. The objective
is not to provide a theory of human or animal emotions so much as to develop
a new art form in which it is easy to produce microworlds containing "believable"
agents which appear to be emotional. The emotion model is embedded in one
module of the overall architecture and emotion is used as a means of creating
more veridical social behavior. The emotion module, called Em, implements
a subset of the Ortony, Collins and Clore cognitive
appraisal theory of emotion, with minor modifications. Objects, other agents,
and current state of the environment are interpreted in terms of liked
and disliked features, behaviors, and current goals respectively and appropriate
emotional state ensues (e.g., happy when goal is satisfied, approach object
that is liked, avoid object that is disliked, etc.). A mechanism exists
for determining which stimuli will cause an emotional reaction by incorporating
a cumulative threshold mechanisms. Thus one strong stimulus or several
weak ones can trigger an emotional reaction. Emotions have decay constants
and may persist over periods of time, depending on the emotion type and
the exact situation encountered.
- Sloman et al. (see
also Cognition
and Affect Project, Univ. of Birmingham, UK)
Sloman et al. use an approach to exploring emotion that is analogous to
Newell's unified theory of cognition (Newell, 1990); that is, the idea
that cognitive mechanisms cannot be investigated in isolation but must
be embedded within an overall architecture that includes mechanisms concerned
with perception and the generation and management of motives and affective
states, in a resource-bounded agent.
Sloman refers to this as a "design-based" approach, exploring
alternative architectures, their requirements and their implications, which
he contrasts with complementary approaches such as empirical, neural, and
philosophical studies. He and his colleagues have proposed a "first-draft"
design-based theory of affect, which is embedded within an overall cognitive
architecture.
Some important emotions typically found in human beings, though possibly
not in all other animals, such as humiliation, excited anticipation, infatuation,
the thrill of discovery, despair, etc. are considered as emergent properties
of the complex control systems comprising the cognitive apparatus and are
not thought to be mediated by separate structures. In fact, Sloman does
not draw a sharp line between cognitive and emotional processing. Sloman's
work views emotions as "dispositional tendencies", tightly coupled
with motivation and the organism's goals. Emotions may or may not be conscious.
The cognitive architecture implementing these ideas includes a hierarchy
of dispositions and implements some of the ideas regarding control and
interruption of cognitive processes found in Simon (1967) also found in
the global signal interrupt theory of Oatley and Johnson (1987).
One subclass of such emergent states (called "perturbances")
can arise when a conflict exists between goals and current state and plans
and behavior must therefore be modified. Other cases can arise when thoughts
about desirable or undesirable states which will happen or might happen,
or might have happened (etc.) tend to disrupt or disturb other mental activities
to which the agent gives higher priority, e.g. during, excited anticipation
or grief, making it hard to concentrate on important tasks.
The hypothesised architecture uses a hybrid symbolic and sub-symbolic modeling
approach and consists of a number of interlinked modules. Some of the sub-symbolic
(automatic, pre-attentive) modules are thought to use very old parts of
the brain shared with many other animals, including highly parallel dedicated
mechanisms reacting directly to external and internal stimulation. Other
resouce-limited modules, comprising an attentive, "management"
system, seem to require symbol manipulating capabilities, re-usable workspaces,
the ability to construct complex (often temporary) structures, evaluate
them and select between them. Additional modules perform "meta-management"
which includes monitoring the behaviour and progress of management processes,
evaluating them, and possibly re-directing them, not always successfully
-- e.g in emotional states!
Most of the ideas are still very sketchy, and very little has been implemented
so far (March 1996), though there are some exploratory implementations
of small subsets, using the "Broad and shallow" design philosophy
of the OZ project to start with, but with the aim of progressively extending
and deepening the designs and the implementations, while progressively
taking account of wider ranges of known phenomena.
- Elliot's Affective Reasoner system (see also Clark
Elliot - Research
problems in the use of a shallow AI model of Personality and emotion)
is another example of a model whose focus is the I/O emulation of affective
processing or on deducing another agent's affective state. These system
focus on mapping situation and agent state variables onto a set of specific
emotions and producing behaviors corresponding to the specific emotion.
The Affective Reasoner has an embedded symbolic AI model which implements
an extended version of the OCC model; that is, it implements a cognitive
appraisal process. The Affective Reasoner functions in the context of an
agent world, where each agent has a set of symbolic appraisal frames that
contain the agent's goals, preferences, principles guiding behavior, as
well as current moods. A specific emotional behavior is then derived in
a particular situation based on these appraisal frames (e.g., if agent's
goals are frustrated agent may become angry and display "angry"
behavior). Agents are capable of displaying emotions, changing their emotional
state, and able to derive other agents' emotions from their observed behaviors.
The Affective Reasoner includes multi-media capabilities (e.g., speech
recognition, text-to-speech translation, and the display of cartoon facial
expressions for variety of emotions). The applications of the Affective
Reasoner include: affective user modeling to improve human-computer interaction,
testing of appraisal theories of emotion, and the development of more believable
characters in computer games.
Task-Level
Models
Models in this category tend to reflect the output model
perspective, in that their focus is often on enhancing system performance
on a particular problem-solving task.
- Dyer's BORIS and OpEd
(1987) focus on natural language understanding and both are examples
of the output model approach. Both models implement a model of "cold"
cognition, in that the cognitive processing itself is not affected by emotion.
Both use symbolic representations of emotions to reason about emotional
states and intentions in stories. BORIS is a natural language understanding
program which has the capability to understand characters' emotional states
and can infer an emotional state from the text of a story. BORIS' knowledge
base consists of symbolic structures representing emotions as five-slot
frames. The theoretical basis underlying the model comes from the cognitive
appraisal theory of Solomon (1980), who defines emotions as specific elements
of the set defined by the cross-product of the organism's beliefs, goals,
and states of arousal. Arousal can be either negative or positive, depending
on the status of the organism's goals and expectations about their satisfaction.
OpEd is an extension of BORIS where beliefs are first class objects and
OpEd can thus reason more effectively over the system's belief states.
- Allen(1993) which implements an aspect of Frijda's theory of
emotion (1986) in a distributed agent environment. Frijda's theory assigns
emotions the function of watching over the individual's goals and making
sure that behavior contributing to their satisfaction. The agent architecture
consists of several components, which communicate via a blackboard. The
primary components involved in emotion processing are the analyzer,
which processes inputs from the perceptual and cognitive systems, and provides
data to a comparator, which compares the input with the current
goals. The highest-level goal is then selected for action.
Mechanism-Level
Models
The models in this category attempt to emulate some aspects
of the mechanisms involved in emotional processing and are therefore at
the process-level end of the modeling approach spectrum. They include
symbolic, connectionist, and hybrid connectionist-symbolic approaches.
We divide these models into those addressing higher-level phenomena,
such as mood congruent recall, the effect of emotion on performance, and
the cognitive appraisal process itself, and lower-level phenomena, such
as classical conditioning, connectionist models of the interaction of cognition
and affect and multiple processing systems (e.g., implicit and explicit
processing), and network models of psychopathology. The latter tend to
be implemented using connectionist architectures.
- Dyer's DAYDREAMER (1987) models emotional states and their influence
on memory, learning, planning, and thought generation. The model attempts
to represent "hot" cognition, by allowing an effect of emotion
on the cognitive processing itself. The basic representational formalism
is symbolic, consisting of conceptual dependency structures made up of
scripts, plans, goals, and primitive acts and containing plot units, which
are used to provide larger semantic framework for generating daydreams.
Different emotions are expressed in terms of a scalar representing positive
or negative state of arousal. The overall emotional state is a function
of the polarity of the individual goals which have different weights associated
with them.
Scherer(1993)
(see also Geneva
Emotion Research Group) has constructed and implemented a computation
model of the cognitive appraisal theory. His model is essentially a knowledge-based
system which takes as its input a description of a situation in terms of
15 "appraisal" dimensions and matches them to an emotion defined
in terms of 14 prototypical emotion components. Both the appraisal and
emotion dimensions are represented as feature vectors and the matching
process uses Euclidian distance between the input vector and a target emotion
vector.
- Chwelos and Oatley (1994) criticize the Scherer model, while
acknowledging its contribution to emotion research. Specifically, they
argue against the vector-space approach and criticize its inability to
generate "no emotion" when no match is found, as well as its
inability to map several combinations of appraisals onto a single emotion.
They discuss a variety of other approaches to implementing the appraisal
process, each one based on a different computational formalism, including
decision trees, rules, pattern matching, and connectionist models. They
conclude that the connectionist approach appears the most promising, providing
capability for approximate matches, generalization, and extraction of higher
order patterns.
- ACRES Frijda
and Swagerman (1987) implemented a cognitive appraisal process based
on Frijda's theory of emotion elicitation. As is the case with all cognitive
appraisal theory models, this model has a matching mechanism whereby features
of a situation are mapped onto a set of output emotions. The following
abstract has been contributed by Paul den Dulk from the University of Amsterdam:
"Emotions can be regarded as the manifestations of a system that realises
multiple concerns and operates in an uncertain environment. Taking the
concern realisation function as a starting point, it is argued that the
major phenomena of emotion follow from considerations of what properties
a subsystem implementing that function should have. The major phenomena
are: the existence of the feelings of pleasure and pain, the importance
of cognitive or appraisal variables, the presence of innate, pre-programmed
behaviours as well as of complex constructed plans for achieving emotion
goals, and the occurrence of behavioural interruption, disturbance and
impulse-like priority of emotional goals. The system properties underlying
these phenomena are facilities for relevance detection of events with regard
to the multiple concerns, availability of relevance signals that can be
recognised by the action system, and facilities for control precedence,
or flexible goal priority ordering and shift. A computer program, ACRES,
is described that is built upon the specifications provided by this emotion
theory. It operates in an operator-machine interaction involving the task
of executing a knowledge manipulation task (the knowledge domain happens
to be about emotions). ACRES responds emotionally when one of his concerns
(e.g. errorless input, being kept busy, receiving varied input, not being
killed) is touched upon. Responses are social signals, shifts in resource
allocations to the operator, interruption of current task-directed processing,
and refusal to accept instructions. His flow of behaviour shows much of
the preference-based predictability, response interference, goalshifts,
and social signalling of human and animal emotional behaviour."
- Ortony, Clore, and Collins (1988) The model's
focus is on determining an emotion from the features of a situation coupled
with the organism's beliefs and goals and the model is thus essentially
a knowledge-based system that reasons over the domain of emotions and emotion-inducing
features of a situation. A major limitation of this model is its limited
capability for emulating "hot" cognitions; i.e., for emotions
to affect cognition. While acknowledging the importance of physiology and
behavior on emotional processing, the authors ignore these factors in their
model, focusing instead solely on the cognitive structures and mechanisms
mediating the appraisal of external stimuli (i.e., events, agents, objects)
leading to affective reactions. While not implemented by the authors, this
model, and its variations, has served as the basis for implementation of
several other computational models, including the work of Bates and colleagues
discussed above. Emotions are divided into three mutually exclusive categories,
depending on the stimulus causing the emotion: 1) those induced by events;
2) those induced by agents; and 3) those induced by objects. OCC construct
complex taxonomies of these entities, where each category of stimuli induces
certain types of emotions via other intervening structures and variables.
Thus events are related to goals, agents to standards, and objects to attitudes.
The only point in the model where a feedback exists from the emotional
system to the cognitive system in an apparent ability of the model to enable
a certain mood to modify thresholds for reacting to events and experiencing
affective reactions (e.g., a good mood will increase the threshold for
irritation to some irritating event).
Lower Level Phenomena
- Araujo (1993) has implemented a connectionist model of emotional
processing which emulates two observed psychological phenomena: the effect
of emotional state on performance and the effect of emotional state on
memory and recall. The model consists of two interacting connectionist
networks: one for emotional processing (EN) and one for cognitive processing
(CN). EN is a 3-layer, one-shot recurrent network which calculates valence
and arousal for each stimulus. CN is also a recurrent network for auto
and hetero associative tasks. A distinguishing feature of this model is
that EN can modify parameters controlling the processing in the cognitive
net, which influence speed and accuracy of the vector retrieval. The theoretical
view of emotion implemented in this model is essentially that of LeDoux
(1989), who posits two separate but interacting systems mediating cognitive
and affective processing. Each subsystem processes different aspects of
the stimulus and each has distinct processing characteristics; the affective
system is faster, processes features of stimuli relevant to the organism's
survival, and generates a simple approach/avoidance output. Cognitive system
is slower, processes many more features, is capable of a high-degree of
differentiation, and relates stimulus features to other information, not
limited to the organism's goals. The fundamental emotional variables are
level arousal (high or low) and valence (positive or negative).
- Phaf
and colleagues (see also Institute
for Emotion and Motivation, University of Amsterdam, Netherlands) have
been constructing models of affective priming and the emotional Stroop
task using primarily connectionist methodologies.
Computational Models of Emotion
Recognition
Addressing the social role of emotion indirectly,
at a different level of abstraction, a number of efforts have focused on
the recognition of emotional states from facial expressions. Padgett, Cottrell
and Adolphs (1996) have built a 2-layer feed-forward connectionist model
that recognizes six basic emotions (happiness, surprise, sadness, anger,
fear, and disgust) from static face images. The net is trained on blocks
of features from the most expressive parts of the face: eyes and mouth.
The model is able to perform categorical perception of the different emotions
and preliminary experiments with human subjects indicate that it successfully
emulates human perceptual behavior. Another example of this type of work
is the Facial Expression Analysis Tool (FEAT) and Facial Action Composing
Environment (FACE), developed at the University
of Geneva (Kaiser et al., 1994). These systems use connectionist models
to construct mappings between emotional states and distinct configurations
of facial muscles. Fellous (1995) has explored principal component analysis
and discriminant analysis in inferring emotional expressions from a series
of five facial measurements. A number of more mathematically-oriented approaches
to facial expression recognition are described in a survey article by Samal
and Iyengar (1992).
Related Sites: MIT
Affective Computing Group (Roz Picard),University
of Geneva, Berkeley
Psychophysiology Laboratory, UCSC
Editors: Eva Hudlicka [psychometrixassociates.com
]
Contributors:
A. Sloman, Cognition
and Affect Project, Univ. of Birmingham, UK
Paul den Dulk (dendulk@psy.uva.nl)
Please
send us your comments.
...Navigate
The Emotion Site