How Emotions Are Made by Lisa Feldman Barrett

Psychologists and neuroscientists have a pretty good grasp on the production of emotional experience. It is a universal, cross-cultural method of communicating our internal states to others; Darwin claimed that producing specific emotional facial expressions (happiness, sadness, fear, disgust and anger) and our interpretation of other people’s facial expressions is the same across Homo sapiens. Many more studies from pioneering emotions researchers such as Paul Ekman have backed up the cross-cultural identification of emotions from pictures of human faces not only from Western cultures but also in Papua New Guinea tribespeople who had never interacted with the West. ‘Classification of Inner States’ and ‘Emotions’ are two of the human universal traits collated by DE Brown in Human Universals and quoted approvingly in Stephen Pinker’s The Blank Slate.

The premise of How Emotions Are Made is that this consensus is wrong.

Lisa Feldman Barrett is a prominent research psychologist. Her website claims she is in the top 1% cited scientists in the world with all the academic reach that entails – there’s the hundreds of research articles in peer-reviewed journals, the TED Talk and the mass-market popular science books. How Emotions Are Made is one of the latter, and leads us through recent research into the neuroscience of emotions while discussing its implications. In this review I want to go through her argument (including some of its later extensions in her scholarly work) and discuss what this means for our understanding of emotions, as well as how this might relate to particular mental health disorders.

Can We Universally Recognise Emotional Facial Expressions?

Feldman Barrett starts by pointing out that there are some problems with the cross-cultural studies of emotional archetypes completed by Ekman and others. The first questions were raised in the 1990s by psychologists like Andrew Ortony and James Russell, who pointed out the large dependence of Ekman on the method in which the questions are asked. Compare the following two methods of asking people to identify the emotion depicted (from Paul Ekman’s work in Papua New Guinea):

From https://www.paulekman.com/resources/universal-facial-expressions/

If you have multiple emotions to choose from, then people across the world tend to agree when shown faces like these. But here is another way of testing people’s understanding of emotions on faces, adapted from an example Feldman Barrett uses in the book. What emotion is being displayed here? Decide now before you read on.

The rates of ‘universality’ of recognition of facial emotion drops sharply with more open-ended questions, and most damningly, a repeat visit to Papua New Guinea in 2013 from psychologist Carlos Crivellari showed that, when open-ended questions were posed to tribespeople in their own language, only the happy face was endorsed as such by more than half the sample, with the other supposedly universal emotional faces described with many different overlapping terms (for instance, disgust was interpreted as anger, sadness or fear).

Similarly, Feldman Barrett’s group re-created Ekman’s facial photos with professional actors (she mentions Martin Landau and Melissa Leo), whom you would suppose would be good at producing easily recognisable emotions on demand, and asked people to identify the emotion expressed in the photo both with and without a contextual statement. A contextual statement might be something like “this man has just witnessed a shooting”, and 66% of people who have read the statement identify Martin Landau as being scared – but this is much higher than if the viewer looks at the photo by itself (in this situation, the majority of the latter group identified him as surprised, with only 38% identifying him as scared).

What if you have more context for the emotion displayed above? Here is the rest of the picture:

Did you pick “thrilled by victory” as the emotion from the first picture? Even if you identified the cropped picture as one of Serena Williams, it would be hard to tell whether she had lost a point or won one (in fact, you could still argue the toss either way even with the full picture). Without context such as a knowledge of the situation or by knowing the individual person very well, our ability to interpret ‘universal’ emotions is very shaky.

Emotional Fingerprints (or Lack Thereof) and the Construction of Emotion

We can’t reliably read emotions on other people’s faces without context, but what if we look at more objective measures? Researchers have hooked subjects up to facial electromyography to objectively measure the movement of facial muscles when particular stated emotions are displayed – they find that there is little overlap between one person’s surprised face and another’s. The most they can distinguish is a pleasant feeling from an unpleasant one.

And as without, so within. A recent meta-analysis failed to find bodily correlates of fear, anger, sadness and disgust within the autonomic nervous system (such as heart rate, skin conductance/sweating, blood pressure, and adrenaline rates). Even more recent studies from rival researchers – researchers who are wedded to the idea of universality of emotions – show huge overlap in how different emotions are experienced in the body. How do you robustly distinguish anger from surprise in the following figure, or jealousy from fear, especially in a single person? The most you seem to be able to do is to distinguish all those ‘hotter’ emotions from another ‘colder’ cluster of depression or sadness.

Over 5000 people clicked on a computer model of the areas of the body that feel ‘activated’ in particular emotions (warmer colours) or ‘deactivated’ (cooler colours) to produce this composite. Note the overlap, except in depression/sadness, which appears to be its own cluster. From https://acris.aalto.fi/ws/portalfiles/portal/36173502/Bodily_maps_of_emotions_are_culturally_universal.pdf

Looking for correlates to emotions in the brain is the same story – even the amygdala (the area most people know as the brain’s ‘fear centre’) has been shown to ‘light up’ on fMRI with any novel situation, not just frightening ones. Any efforts to find a particular brain region, or brain network, that reliably correlates with people’s expression of a particular emotion has proven fruitless.

Why is it so difficult to find a reliable marker in the body that correlates to an identifiable emotion? Feldman Barrett’s idea is that emotions are not natural types and therefore looking for their ‘fingerprints’ is useless (or, at the very least, needs to be considered in a wider way). Instead, they are constructed in the same way that our brain constructs other experiences, via Bayesian mechanisms.

Regular readers of ACX won’t need to be told about Bayesian processes in the brain, but in the general population this view is not common. Most people assume that an otherwise unoccupied brain gets a stimulus, ‘processes’ the stimulus in some undefined fashion, then produces a response. This is an oversimplification based on left-over relics of behaviourism: there is increasing recognition that the brain works more by producing a Bayesian simulation/prediction of the world around it, which it then tweaks with incoming sense data to update on its priors. To tweak however, you need to be able to recognise the failure of your prior world simulation to predict the future, and update accordingly (ie, you need to process your prediction error).

The important part here is that interpretation of incoming sense data is based on anything that produces your priors – previous experience, your current mood state and the context you are in. Not only does this conception produce many of the more basic phenomena of consciousness such as ‘filling in’ of visual information but, as Feldman Barrett discusses, produces the nuances of different emotions. The physical manifestations of emotion are non-specific, and our interpretation of them depends on context.

For instance, Feldman Barrett tells the story of a date she went on as a young woman: she didn’t feel she was going to get on well with her date, but during coffee she began to feel some flushing in her face and butterflies in her stomach. She began to ask herself “Perhaps there are some feelings there?” This lasted right up until she got home and began vomiting, having previously contracted the flu. She asks, reasonably, how can someone mistake an infection for the first flutterings of young love?

Her answer lies in the idea that Bayesian management of prediction error isn’t just occurring with sensory data from the world around us; it’s occurring with our internal sensory data and the brain’s model of bodily function– a concept most commonly called interoception, and that Feldman Barrett combines with the concept of the brain’s regulation of bodily energy in what she calls allostasis.

From https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6054486/ ACC, anterior cingulate cortex; AIC, anterior insular cortex; MC, metacognitive layer; MIC, midinsular cortex; OFC, orbitofrontal cortex; PE, prediction error; PIC, posterior insular cortex; SGC, subgenual cortex.

How is interoception linked with emotion? Basic pleasant or unpleasant bodily sensations are interpreted by the brain in the social context, producing psychological states we identify with prior labels that are produced, in part, by the sociocultural context in which we grew up. That is, an upset tummy and flushing occurring on a date might be interpreted as love or lust; or, in another evocative example Feldman Barrett gives, perfectly edible baby food smeared on a baby’s nappy will produce nausea and disgust when people are asked to smell it, with the normally neutral/pleasant smell overcome by the prior of “I’m sniffing a nappy”. The famous example of judges handing down harsher sentences just before lunch is explained in this context as well: the unpleasant interoceptive sensations of hunger are interpreted in the light of the situation they find themselves in (a convicted criminal awaiting their judgement).

Emotions can even be primed (that is, appropriate priors can be set up), as shown in some replicated experiments that examine people who are given air with increased levels of carbon dioxide to breathe. C02 will indirectly produce an increase in sympathetic activity (releasing adrenaline), but if you preface the experiment by telling one group “this stimulus makes people feel tense, like feeling excited” and the other group “this stimulus makes people feel tense, like feeling anxious”, then the evoked emotions break along the expected lines.

The Neuroscience of Emotion

In the book (and in her subsequent work) Feldman Barrett takes pains to dispel many of the classical ideas about how the brain works in the production of emotion. Most notably, her group tries to debunk the idea that identifiable psychological functions within the brain are localised to specific regions or specific neural networks, as well as the notion of a one-to-one correspondence between psychological functions and particular neural networks.

In contrast, concurrently with other work from Pessoa amongst others, the hallmark of the neural architecture of emotion (and of psychological states in general) is that signals can vary from person to person and often involve whole-brain signals that incorporate areas of the brain that do not seem directly connected to the classical neural networks we know and love. These networks are still important – the link between the Default Mode Network and the Salience Network appears relevant in much of our interoception of bodily models – but they are not sufficient. Any one specific neural network state you could name may not even be necessary for a particular psychological state, as there is also significant degeneracy of neuronal inputs into psychological states (that is, many network states can produce similar psychological states). This also explains why it is difficult to interpret the results from functional imaging studies into the links between brain networks and single psychological states: the external environment, current bodily state and the task being performed all interact in complex ways to make neural activity hard to measure consistently, both between subjects and between time points.

From https://www.cell.com/trends/cognitive-sciences/fulltext/S1364-6613(22)00332-1

But this doesn’t mean that there are no constraints on how people are wired, and the theory of constructed emotion is not purely a theory of social construction. The architecture of the brain and its link to the body is similar across people, and there are only so many ways that neural networks can form. Unless you are a masochist, pleasant and unpleasant interoceptive feelings are generally linked to stimuli that signal the health or danger/damage to the organism, respectively – context-specific labelling of emotions is one of the ways we make sense of how our brains have evolved to monitor our bodily systems that keep us alive.

How are Emotions Constructed?

In infancy, babies can show statistical decision-making (for instance, crawling to a box that is more likely to have their preferred toy) but can also, through experience, link more abstract concepts to objects in the world. The more abstract the concept, the longer it takes to sink in and the further along in neurodevelopment children have to be to ‘get it’, but the process is faster and more reliable when the concept is linked to words spoken by an adult. It is difficult, for example, to learn the concept of money without a shorthand piece of language; looking at piles of gold, credit cards and paper bills doesn’t immediately suggest a common category, but mentioning to a child that each is a form of money allows them to make the connection. Research with young children has, in proof of concept, taught the concept of a “Blik”, by showing them a series of otherwise unrelated objects, telling them “this is a blik” and playing a beeping sound after each presentation. After only a few repetitions, being shown anything identified as a blik will lead the children to expect the beep.

In the same way, adults teach the concept of an emotion to children by naming it: “You must be angry” can be a response to flinging unwanted food, someone who has just hit another child, or to someone who is red in the face; in these situations the child learns to group all of these disparate experiences (which, remember, don’t have common neural correlates) into a category that allows easy description to others and identification of similar sensations when they occur again.

If our concept of emotions and the language we use to speak about them is dependent on how we are taught by adults and others around us, it follows that different cultures have different ways of expressing emotion. Some come into English from other languages and some remain culture-bound; Feldman Barrett points out, for example, that Russian has two words for anger (serdit’sia and zlit’sia), German has three and Mandarin five. Does that mean that people in different cultures are thinking differently about the emotion of anger? Or, put another way, could you feel – really feel – the emotion of schadenfreude before you learned the term? Well, yes. You don’t need the word, but you do need the concept, which you can construct from having a number of similar experiences that you can categorise together (sometimes unconsciously). Words are just a good shorthand way of categorising concepts - Feldman Barrett invents one for us in the book, “chiplessness”, the feeling of wanting chips but not having any immediately to hand. Chiplessness, if integrated into the the wider culture in such a way that most people know what the word is referring to, is in this telling at least as valid as feeling (for example) flabbergasted.

People vary a lot in their ability to identify the emotional concepts they are feeling. If you feel a specific constellation of bodily symptoms (increased heart rate, restlessness, sweatiness) you might call it anxiety, but some call it sadness, some call it excitement and others simply report their physical symptoms without assigning an emotion at all. Feldman Barrett describes this with a specific term, ‘emotional granularity.’ Individuals with high emotional granularity have hundreds of emotional concepts they can describe, many of these concepts coming with their own words (they can, for example, comment on feeling aggravated or disgruntled, and mean two different things).

There is a range of granularity and sophistication in emotional language, and people in the Western context (but not necessarily in the Eastern) whom you would expect to have developed some of this vocabulary but haven’t, are labelled as alexithymic – these individuals appear not only to lack the vocabulary to describe emotions, they don’t appear to feel the underlying emotion either. Such people are the ones who are more likely to ‘somatise’; that is, they describe physical symptoms such as chest or abdominal pain rather than identifiable emotions.

What if you can’t internalise or learn emotional concepts in the first place? If there are deficits in the process of learning abstract concepts or an inability to produce an effective predictive model of the world, you would have difficulties in managing sensory input, develop resultant anxiety and a desire for externally-enforced routines, along with a deficit in feeling emotions and recognising emotions in others – in other words, you would be on the autism spectrum.

Overall, the range of emotional granularity might look a little like this (facetious diagram is mine):

What Is Emotion For?

Feldman Barrett isn’t the first to highlight the importance of interoception and its role in embodied cognition. Damasio, amongst others, covered similar ground in his research and its pop-science translations (Descartes’ Error and The Feeling of What Happens, amongst others). But Feldman Barrett provides an easy-to-follow way to describe why interoception is so important, using the concept of bodily energy budgeting.

While the specific names and labels of emotions might vary, their role can be explained by how well they allow the brain to model what is going on inside the body, both in terms of energy expenditure and to predict how the rate of energy expenditure may change. Predictive processing and the internal bodily (interoceptive) model it creates is represented through affect, although the purpose of interoception is not the production of affect, it’s the regulation of the ‘body budget.’

Affect, nonetheless, is one of the fundamental features of consciousness (or at least self-consciousness). It varies in two main dimensions – level of arousal (calm through to agitated) and pleasure/displeasure, and all our emotion labels can be situated somewhere on these dimensions.

From https://commons.wikimedia.org/wiki/File:Valence-Arousal_Circumplex.jpg

If the brain is predicting an increase in energy requirements, then it will ramp up ‘agitating’ responses such as those provided by the adrenaline-mediated sympathetic nervous system. Adrenaline produces predictable physiological effects – elevated heart rate, sweating, diversion of bloodflow to the muscles, churning stomach, dry mouth, and so on (these would produce the ‘hot’ colours in the body map figure above). These effects in turn are fed back to the brain which then interprets them according to its priors. This effect occurs whether the predictions are correct or not, and while Feldman Barrett only touches briefly on how this might explain various mental disorders you can already see the outlines of a common way the vicious cycle of a panic attack might occur: the brain erroneously predicts the need for action, the appropriate adrenaline response occurs, there is an interoceptive feedback loop that reinforces the agitation, which is then interpreted according to whatever priors the person has developed about their ability to deal with the agitation (common panic attack priors include ‘when I feel this way, I’m about to die’ or ‘when I feel this way, I’m going to go mad’).

Conversely, a predictive model that indicates a need to conserve energy might produce a bodily response that reduces unnecessary movement or activity (producing the ‘cold’ colours on the body map figure). But if this model is inaccurate, and your priors become trapped you might withdraw unnecessarily and produce a vicious cycle that worsens the anergia and solidifies any prior beliefs that are consistent with the need for isolation – this is a common presentation of depression.

What Does This All Mean?

Political scientist Yascha Mounk has noted that books about broad political trends usually have a “Chapter 10 problem”: they spend nine chapters talking about their theory of the world, and then in the final chapter end up admitting that they don’t really know what to do about it. Popular science books generally have the opposite problem. They usually summarise the underlying science very well, and then have a chapter or series of chapters at the end where their sights turn to extrapolating their theory wildly into speculations that apply to broad swathes of human nature and philosophy.

Feldman Barrett doesn’t really wait until the last chapter or two to let rip here; Chapter 8 of 13 is entitled “A New View of Human Nature.” Here she claims that the theory of emotional construction means that our emotions and hence many of our reactions are malleable and suggests that her theory gives hope for managing entrenched hate in multigenerational conflicts such as the Israel-Palestine crisis. This seems… simplistic. Less grandiose is the idea that this theory jibes well with models of neuroplasticity and gives hope that people can effectively change their concepts and therefore change their minds. If so, Feldman Barrett has, after many years working in psychology, successfully invented the concept of therapy.

There are more interesting questions that arise from considering emotions as constructed. In the book, Feldman Barrett describes going to the doctor and feeling tired, anergic and frazzled. The doctor suggested a diagnosis of depression, which Feldman Barrett rejected, only to be told “maybe you’re depressed but don’t know it”. She points out, rightly in my view, that an approach such as this has the potential to label her individual psychological state in a way that is powerful but not necessarily accurate – or useful. And it has the potential to define her experience in ways that then alter her priors. She counsels caution from health professionals in allocating specific emotions to patients who report various sensory experiences; conversely, she suggests that when people who come into a doctor’s office claiming depression (or any other emotion), the doctor asks exactly how things feel in the patient’s body rather than taking the emotion at face value.

Another claim for this theory is that you can only express emotions you know about, and this has some implications for how we should be talking to our children about feelings. One of the jobs of parenting has always been to contextualise and teach children about how to manage their emotions, and while it would be tempting to think that you could serve up a set of bespoke, positive emotional concepts to teach your child, this doesn’t really make sense – children need to be able to make sense of negative affect as well as positive, and the social construction of specific emotions doesn’t mean that negative feelings can be ignored. Rather, I think the lesson here is that trying to increase the emotional granularity of children about negative emotions (that is, teaching and modelling as many emotional concepts as they can manage) allows them to conceptualise different feeling states and develop nuanced responses. You want someone to be able to distinguish aggravation from disgruntlement, as these might have more adaptive responses in different circumstances.

Feldman Barrett asks the question: If you don’t know about any emotions do you feel them? She answers that not being socially primed into particular concepts will make particular emotions more difficult to express, but I think that before we start recruiting the anti-memetic police we need to look more closely at the fate of people with alexithymia or those on the autism spectrum, who seem to still feel unpleasant and agitated, but don’t know why. If you can’t identify emotions well, then the responses become simpler or more inflexible, and it’s unclear whether that is a better option.

(Just to be clear, I don’t think Feldman Barrett is re-suggesting the ‘refrigerator mother’ theory of autism, which is the idea that inadequate validation or parenting leads to autism. Rather, any combination of genetic risk or early developmental insults such as birth trauma that impair the ability for someone to develop emotional granularity can predispose people to some behaviours that lie on the spectrum, regardless of how emotionally in touch the parents may be).

In general, Feldman Barrett gives you what you want out of a mass-market popular science book. She leads you by the nose through well-validated, peer-reviewed, replicated research that overturns some of the popular stereotypes about an area of science. She covers the deficiencies of the classical view of emotions, the surprising difficulty in pinning down the concept of a single emotion such as anger or fear (or any of its supposed neural correlates) and of energy management and the ways this helps to produce subjective experiences. She raises interesting questions which are hopelessly extrapolated to solutions in ways that almost seem reasonable while you’re reading them (Israel-Palestine, solved!) and, by extension, make you excited about the research and what it might mean. As Feldman Barrett herself states, popular science writing is not the same as the underlying research and if you are excited, then going to the academic literature can be useful to answer more granular questions (although I’ll warn you, many of the articles are pretty turgidly written).

For ACX readers this book helps show us that there is a reasonable amount of evidence that social construction of psychological experiences happens at several levels of phenomena. It bridges the gap between constructional ideas in more basic parts of consciousness studies (the emergent nature of qualia, visual ‘filling in’) and the discussions of social construction of psychiatric diagnosis. It allows us to think about recent discussions about culturally-created mental disorders (as discussed in Scott’s recent Geography of Madness review) as a functional level above emotional concepts that are culturally-created on a scaffolding of neural networks. And it works as a good recommendation for those who want to introduce themselves or others to the ideas of predictive processing and the Bayesian processes within the brain.