Sign-tracking Sucks
Put a rat in a cage. Show it a lever. A few seconds later, regardless of what it does, give it some food. Repeat a couple dozen times a day for a week and the rat will learn to associate the lever with food. Some rats go over to where the food will come as soon as they see the lever. Those rats are goal-trackers. The rest of the rats will start to play with the lever and maybe chew on it once they see the lever. And then once the food appears they'll go over to that part of the cage and eat it. Those rats are sign-trackers. Sign-tracking entails the release of stress hormones, and its pointless because the sign-trackers don't get any more food than the goal-trackers. Sign-tracking sucks; zero out of five stars; do not recommend.
Imagine that you come home and your dog runs up and jumps on you and licks your face, while your cat goes to the food bowl and waits expectantly. Your cat is exemplifying goal-tracking, because food is a cat’s ultimate goal; your dog is exemplifying sign-tracking, because you are the sign that often precedes food.
“Raccoons were trained to deposit a wooden coin through a slot in order to obtain a food reward. The raccoons initially performed this task without hesitation, but with further training seemed unable to let go of the coin, spending several minutes compulsively handling it with their forepaws—chewing, licking, rubbing and washing the coin—as if they were trying to clean a morsel of food —and repeatedly putting the coin into the slot but then pulling it back out without releasing it. The coin itself appeared to have great incentive value, as the raccoons were very reluctant to give it up, even though holding onto it delayed or even prevented receipt of actual food.” [ref] Male quails repeatedly seeing a pillow followed by a female in heat will learn to hump the pillow, even if the door to the female is at the far end of a long skinny cage and their fetishism delays access to real sex. Meanwhile, rats will cower against the opposite wall of their cage when faced with a lever that’s been repeatedly paired with extremely salty water.
Horse trainers will repeatedly pair food with a clicking sound, and then whenever they want to reward the horse, they can just click. And the horse will be more likely to do a trick that earned them the clicking sound. A psychologist would say that the clicking is a conditioned reinforcer, and the new trick is reinforced behavior. Using a conditioned reinforcer makes it much easier to train horses because if you had to give them food every time, you couldn’t train them after you had already rewarded them and they were full. This works because the sign itself can function as a reward, and continues to motivate animals even if it is no longer followed by a reward. (In contrast, goal-trackers will quickly grow bored with their food bowl if you stop putting food in it.)
Sign-tracking in Humans
Researchers can train smokers to click on certain shapes using either cigarettes or money—either reward works just as well. In smokers, cigarettes are almost certianly a conditioned reinforcer, since they have been repeatedly followed by the intoxication of smoking. We can infer that money is also a conditioned reinforcer: it produces the same behavior as other conditioned reinforcers; it increases blood flow to all the same brain regions that show increases with food, sex, alcohol, cocaine and electric shocks; everyone has experience using money to buy rewards. Once we know that money is a conditioned reinforcer, we can invoke sign-tracking to explain greed, since sign-tracking implies that people will value money itself and keep seeking money even if there's nothing in particular that they want to buy.
There’s another experiment showing how humans pursue purely symbolic rewards. Put identical soda in bottles labelled with either a triangle or a square, and then repeatedly show children a triangle followed by a crying face and a square followed by a smiling face. Kids will try the square soda first and say, after trying both, that they like it more. The same thing happens if you repeatedly show kids a triangle together with a crying face and a square together with (rather than followed by) a smiling face, which is how advertising works. By extension, we can infer that you should be a able to reinforce a behavior just by showing people a smiley after they’ve performed a new trick. That’s how social media works: if you post something, and people react with smileys, then next time you’ll be more likely to post. Tech firms can train you without ever giving you a physical reward. How cheap and convenient for them!
If we’re confident that money, cigarettes and smiling faces are conditioned reinforcers, then let’s indulge in a bit speculation (which usually leads to pleasant results!) as to what other everyday signs might be conditioned reinforcers.
When you were a baby, probably your mom said nonsense in the voice that people use for talking to babies and then would feed you, so you learned to associate your mom's voice and baby talk with food. Then your mom would be able to reward you for good behavior just by baby talking to you. She wouldn't actually have to reward and punish you physically all the time. From there you learned to associate specific words with reward. Maybe you would associate “Good job!” with reward because your mom would say it in baby talk, and your mom's baby talk voice is already associated with food. And then from there now other people have the ability to shape your behavior because they can reward you only by saying “Good job!”. Imagine how hard it would be to raise children if you couldn't reward and punish them just with words and body language. As soon as they were full and no longer interested in food you would lose the ability to teach them anything.
If you put on clothes and walk down the street and go to the store and buy something, all of that social etiquette was taught to you by your parents: what kind of clothes to wear, which part of the street to walk on, how to pay for things, what kind of food is good, which things are food, how you should remove the packaging without eating it, and on and on and on. A huge amount of our daily repertoire of behavior we normally don't think about probably is reinforced behavior. In a addition, we have large repertoires of conditioned reinforcers: others can influence our behavior by looking, staring, glaring or gazing at us; by praising us or shouting at us; by wearing an athoritative uniform or a pretty dress.
Being motivated by conditioned reinforcers allowed you to learn the microskills of daily life, but sign-tracking is stresful, pointless, and deceptive.
Sign-tracking is Stressful
The critical experiment which demonstrates the stressful nature of sign-tracking involved lever-pressing in rats. In the experimental group, the rats saw a lever, and a few seconds later food dropped into a bowl, regardless of what the rat did. This sequence was repeated dozens of times. The control group saw the lever and food just as often, but at random times, so they formed no association between the lever and food. All of the rats had low concentrations of stress hormones before the experiment. Afterwards, only the experimental group did. A seperate experiment by the same authors found that in a group of rats all given the same conditioning, only the sign-trackers were stressed and the goal-trackers were relaxed. Additionally, pharmaceutically blocking the activity of stress hormones turns all rats into goal-trackers. These results demonstrate that sign-tracking itself is stresful, and that rats are able to find and eat food in a relaxed manner as long as they don’t sign-track.
This experimental work aligns with anecdata. Stressed people are more likely to smoke, drink, or take drugs. The same brain circuit for craving underlies both addiction and sign-tracking, and addicts are likely to relapse when they encounter a sign associated with their habit. If someone used to always drink red wine and smoke, then they’ll be prone to start smoking again if they’re stressed and drinking red wine. Stock brokers spend all day pursuing big money rewards. By reputation, brokers are crack addicts and super stressed.
Punishment (trauma) causes sign-tracking that’s a mirror image of sign-tracking for rewards. You might be nervous around your boss or another figure associated with punishment. When people with PTSD encounter a sign associated with their traumatic past, they stress out so much that it becomes a clinical pathology.
People will often claim that their nervous habits relieve stress, but the experimental work in rats shows that this is unlikely. Nervous habits are simply compulsive and people confabulate a justification for it. Indulging nervous habits further exposes them to reward-associated signs and gets them deeper into a self-reinforcing cycle of stress and signs.
Chronic Stress is Depressing
In the words of entrepeneur and science writer Max Bennett: “All this describes what bodies do in response to short-term stressors—the acute stress response. But most of the ways that stress plagues modern humanity comes from what happens to bodies in response to prolonged stressors —the chronic stress response. … If a nematode is exposed to thirty minutes of a negative stimulus (such as dangerous heat, freezing cold, or toxic chemicals), at first it will exhibit the hallmarks of the acute stress response—it will try to escape, and stress hormones will pause bodily functions. But after just two minutes of no relief from this inescapable stressor, nematodes do something surprising: they give up. The worm stops moving; it stops trying to escape and just lies there. This surprising behavior is, in fact, quite clever: spending energy escaping is worth the cost only if the stimulus is in fact escapable. Otherwise, the worm is more likely to survive if it conserves energy by waiting. Evolution embedded an ancient biochemical failsafe to ensure that an organism did not waste energy trying to escape something that was inescapable; this failsafe was the early seed of chronic stress and depression.
Any consistent, inescapable, or repeating negative stimuli, such as constant pain or prolonged starvation, will shift a nematode brain into a state of chronic stress. Chronic stress isn’t all that different from acute stress; stress hormones and opioids remain elevated, chronically inhibiting digestion, immune response, appetite, and reproduction. But chronic stress differs from acute stress in at least one important way: it turns off arousal [energy] and motivation … [by] activating serotonin. At first glance, this makes no sense: serotonin was supposed to be the satiation and good-feels chemical. But consider the main effect of serotonin: it turns off valence [pleasure and pain] neuron responses and lowers arousal. If you add this to the soup of stress hormones, you get a bizarre yet unfortunately familiar state—numbness. This is, perhaps, the most primitive form of depression, … [which] dulls pain and renders even the most exciting stimuli entirely unmotivating.
…
[Numbness] in animals like nematodes seems to be a trick to preserve energy in the presence of inescapable stressors. Animals no longer respond to stressors, good food smells, or nearby mates. In humans, this ancient system robs its sufferers of the ability to experience pleasure and motivation. This is the blah or blues of depression. And like all affective states, chronic stress persists after the negative stimuli have gone away. Such learned helplessness, where animals stop trying to escape from negatively valenced stimuli, is seen even in … cockroaches, slugs, and fruit flies.
We have invented drugs that hack these ancient systems. The euphoria provided by natural opioids is meant to be reserved for that brief period after a near-death experience. But humans can now indiscriminately trigger this state with nothing more than a pill. This creates a problem. Repeatedly flooding the brain with opioids creates a state of chronic stress when the drug wears off—adaptation is unavoidable. This then traps opioid users in a vicious cycle of relief, adaptation, chronic stress requiring more drugs to get back to baseline, which causes more adaptation and thereby more chronic stress.” [ref]
Chronic stress from any source, not only stressful sign-tracking, causes depression. Yet I mention this link here because it is necessary to explain much of the toxicity of sign-tracking. The stress of sign-tracking alone can explain why phone-addicted Zoomers have anxiety disorders, but at first glance it seems surprising that they would also suffer from depression, especially since anxiety and depression at first seem to be opposites. The link between chronic stress and depression completes the causal chain: phone-addiction is sign-tracking causes chronic stress causes depression. Stress, the relieving end of stress, and depression all reduce libido, so a generation of phone-addicts will also have low fertility.
Compared to goal-trackers, sign-trackers show increased blood flow to a certain brain area. Compared to healthy controls, depressed people show increased blood flow to the same brain area, and getting an injury there cures depression. Sometimes surgeons will intentionally lobotomize that area as a last-resort treatment for extremely resistant cases of depression. This neurological evidence strengthens the case that sign-tracking causes depression.
Linking sign-tracking through chronic stress to depression can also help us explain why hedonistic rockstars keep killing themselves. Rockstars are over their heads in approval and attention, extremely rewarding reinforced conditioners. The surplus of adoration leads not to lasting happiness, but merely reinforces the lust for approval, triggering a cycle of insecurity and anxiety. The resulting stress predisposes rockstars to form a habit from any drug that they try, leading to drug addictions that compound anxiety into chronic stress, depression and eventually suicide. The last word in this section is from the foremost philospher of ennui, Søren Kierkegaard:
“I have just now come from a party where I was its life and soul; witticisms streamed from my lips, everyone laughed and admired me, but I went away-yes, the dash should be as long as the radius of the earth's orbit———————————and wanted to shoot myself.”
Sign-tracking is Pointless
The cyclical nature of sign-tracking is best demonstrated by brain imaging studies of people viewing pornography or getting blow jobs. When men are viewing porn, the usual brain regions that receive increased blood flow for craving anything else also receive increased blood flow. This is the wanting phase of the craving cycle. For both men and women, when getting blow jobs, a subset of the craving network shows increased blood flow, and stress hormones are also released during this stage, just slightly different ones than for wanting. This is the liking stage of the craving cycle. For both men and women, during orgasm all of the brain regions in the craving network receive decreased blood flow, and no stress hormones are released. This is the satisfaction stage of the craving cycle.
A similar cycle happens for foraging rats. During the wanting stage they need to decide which objects are food and which objects are predators so they can avoid the predators and approach the food. During the liking stage, rats put the food in their mouths and they need to decide whether spit it out or to swallow it. Then during the satisfaction stage they’re full and it’s time to take a nap and digest that tasty meal.
Self-help bloggers usually explain how wanting leaves us on an endless hedonic treadmill and so the secret to happiness is to like things more. That can't be true because otherwise you could just solve all of philosophy and religion by taking semaglutide (Ozempic). The bigger problem is that we know from brain imaging studies of the sexual cycle that liking is distinct from subjective pleasure. Orgasmic bliss not only occurs at a different time than liking and wanting, but the craving network of brain regions active during both liking and wanting are specifically deactivated during orgasm, and so cannot be responsible for subjective pleasure.
This same causal principle is confirmed by giving cocaine addicts a hit while they're in a brain scanner. Cocaine directly hacks the brain chemicals that encode liking and wanting and so cocaine doesn't follow the usual pattern of the craving cycle that we've been discussing, but there's still a clear separation between the self-reported high versus self-reported craving. The high comes quite quickly after addicts take a hit and only lasts for a few minutes, while the brain regions in the craving network that subserve liking and wanting are active evenly and continuously for at least half an hour after addicts take a hit.
All this formal experimental work aligns with anecdata. I usually drink a cup of coffee when I wake up, and so waking up is a situation that I have associated with coffee. This morning when I woke up I immediately started to want coffee. After I had brewed a cup and had taken the first sip I tasted the coffee and had to decide whether to spit it out or to swallow it. Fortunately I liked the taste of the coffee and decided to swallow it. After the cup was finished then I was satisfied and had no particular liking or wanting related to coffee. For the next hour or so I had a pleasant tingling sensation. Later in the day the pleasant tingling sensation had faded, but I still had no particular wanting for coffee, which shows how subjective pleasure is different than satisfaction. That is an important caveat to note because, for example, depressed people don't want anything yet are miserable. Merely the absence of wants does not guarantee happiness.
A lot of people wrongly model this using needs, starting from basic physical needs such as food and water and then abstracting from there. The idea is that you have a physical need for a certain amount of water and then once you get that amount of water then you're not thirsty anymore and then you're satisfied and happy. In this wrong model, more abstract things work the same way. Allegedly you need a certain amount of appreciation or self-actualization or whatever and then when these needs are fulfilled then you'll be happy. This is especially used to critique sexual norms by claiming that you have certain level of sexual activity that's healthy and normal and if you reach this level then you'll be happy.
The needs model might actually be true for simple homeostasis requirements like water and caloric intake. But these homeostatic desires are extended through sign-tracking and so they actually are a loop. The more you indulge them the more you reinforce the incentive value of the conditioned reinforcers and the more desire and stress you have. This is similar to a protection racket because paying the mafia makes the problem go away temporarily but if the mafia didn't exist in the first place then you could be safe without even having to pay them. In just the same way, if you didn't think about sex at all or see any signs associated with sex, then you wouldn't have sexual desire and you wouldn't need sex in order satisfy your desires. The same goes for nearly of the conditioned reinforcers that plague our lives.
Average people have many signs attached to food and drink that make them pointless in the same way as purely symbolic desires. You desire particular comfort foods that you ate when you were a child, and like or dislike different flavors and textures based on what associations you have with them. When you go to a restaurant, you don't merely seek vitamins and calories but the restaurant claims association with some ethnic cooking style and you have sign-tracking deriving from the symbolic ethnic information accompanying the food. If instead you eat in, then you'll have to go grocery shopping and you will have to choose between different brands associated with smiling people. You'll have to choose between healthy and ecological food that's associated with girliness and left-wing politics versus American meat that's associated with manliness and right-wing politics. Although you can become full after a meal and the physical aspect of eating narrowly fits into the needs model, food and drink have a lot of symbolic aspects which fit more into the idea of an endless cycle of craving. Sign-tracking for food-related signs can cause obesity when it goes to a pathological extreme, and is stressful and pointless even in moderation.
Sign-tracking is Deceptive
One of the most evil things about sign-tracking is that it’s hard to reason your way out of it because you actually like the signs that you’re tracking. Remember from earlier, there is a study where children liked the soda brand that was associated with smiling faces. To understand the neural mechanisms of this, researchers use a Matrix-like experimental set up. They surgically install tubes from electric pumps through the back of a rat’s head directly into its mouth so that a computer can programatically feed it different flavors. This is a Matrix from the nightmares of BF Skinner, and so obviously you’ll have one tube of rewarding sucrose solution, and another tube for punishment with highly concentrated saline. Further, the rat’s brain is a pin cushion for electrodes which can measure the electrical activity of just a handful of neurons at a time, much more precisely than fMRI studies which can only measure entire brain regions. Using this setup, researchers associate a first neutral flavor with sugar water and a second neutral flavor with saline. Then they present the neutral flavors in isolation and compare the resulting electrical activity to the activity that resulted from actual sugar and salt. Throughout many areas of the craving network, the electrical activity from tasting a flavor that's associated with sugar or salt is the same as the activity from sugar or salt. The rats liked the sugar-associated flavor just as they liked sugar, and disliked the salt-associated flavor just as they disliked salt. When researchers injected the rats with a chemical that depletes salt from the rats’ blood, then they craved salt and suddenly started to like the salt-associated neutral flavor. Just to make the experiment even more dystopian, the researchers killed the rats, liquified samples of brain tissue, and confirmed that patterns of gene expression were similar between sugar-association and salt-association versus sugar and salt. Someone should make a movie where computers harvest our brains to get data.
Rats value the sign in the same way as the goal, and this is also apparent in anecdata from humans. Imagine someone who enjoys the smell of red wine because they’re used to getting tipsy on it. Then one night, they overdo it and puke. They will suddenly find the smell of red wine unpleasant, because now they no longer like red wine.
Smokers begin to like the stink of cigarettes after their smoking habit has gotten going. Italians drink espresso with dessert after meals, and come to like the bitter espresso. If you want to train your child to eat vegetables, try presenting the vegetables in an extremely sweet preparation that your kid likes. Over many meals, gradually reduce the amount of sugar while ensuring that your kid still likes it, until eventually all the sugar is gone and your kid will still like the sweet-associated vegetables, even if they don't actually have any sugar in them anymore.
The ancient Greeks viewed reason and emotion as separate processes, using the metaphor of a charioteer: reason is the driver who has to control the horses that represent our unruly emotions. The same basic intuition has persisted for centuries: the story of Dr. Jekyll and Mr. Hyde portrays the emotional Mr. Hyde as totally separate from the rational Dr. Jekyll. However, in reality, people like things that they are emotionally attached to, and confabulate reasons why those things are good. Even though sign-tracking is stressful and pointless, we do not have a firewalled reasoning system that we can use to control it. Instead, sign-tracking actually changes the values that we use as the basis of our reasoning. Our most well-thought out schemes are usually in the service of approval, money, or some other reinforced conditioner.
Remember the apocryphal story of Phineas Gage who accidentally got a metal rod through his orbitofrontal cortex, a node of the craving network. He famously became more impulsive. The story clearly demonstrates how we don't have a firewalled reasoning and control process, but rather a single collection of values that contains both better-socialized values we label as self-control and antisocial values we label as animal desire. Although Phineas Gage has become something of a Biblical tale for neuroscience, the moral of the story is well supported by recent work by Antonio Damasio and others who have more rigorously studied patients with damaged orbitofrontal cortices. They have confirmed that such patients indeed have terrible judgment.
All of this makes it extremely hard to talk to people about sign-tracking, because people truly like the signs that they track.
Materialist Virtue Ethics
In a university psychology department, somewhere in the world, a trained rat in a cage is chewing a lever and never getting full. The worst part is that it likes it. That's a metaphor for life bleak enough to make Sartre despair. There is an escape, though. Since sign-tracking is stresful, pointless and deceptive, goal-tracking more often will make us easygoing, purposeful, and wise. To goal-track more and become a better person, we need skeptical inquiry into the incentive values attached to objects by sign-tracking. Suspecting the values instilled in us by propagandists and marketers allows us to articulate a virtue ethics grounded in scientific skepticism. It gives us a precise language to explain why more rewards make us less happy and to justify self-restraint of our own surplus enjoyment.
Sign-tracking sucks; zero out of five stars; do not recommend.