I.

My favorite Less Wrong posts are those that name and explain a seemingly familiar concept, and by doing so demolish a prior illusion of understanding. The Evolution of Cooperation by Robert Axelrod predates the Rationalists by a few decades, but it fits the criteria exactly.

The core intuition I had going into the book was pretty simple: even though betraying people can lead to short-term gains, it's almost never the right move in real life.

I fervently believe this to be true and have lived by this axiom for pretty much all my life; I basically just spam cooperation and I’ve been rewarded handsomely for it. But I was never sure exactly why it worked, and so all of my attempts to explain my strategy to others always turned into rants about the fundamental goodness of humanity and the power of friendship. It turns out that these arguments are not very convincing, particularly when directed at pessimists, doomers, and general misanthropes.

The Evolution of Cooperation is essentially a game-theoretic case for my core intuition. Its fundamental insight is pretty simple: the optimal strategy is to cooperate as much as you can without letting yourself get exploited too much.

This slight alteration adds some sorely-needed nuance to my original intuition. For example:

In real life, we frequently interact with the same people over long periods of time. The more we expect to interact with someone in the future, the smarter it is to cooperate with them and build trust.
An ideal strategy is somewhat vulnerable to betrayal. But this is okay: tanking some betrayals allows you to cooperate more often, the benefits of which heavily outweigh the costs of occasional treachery.
Still, if you gain a reputation for being a pushover, evil people will notice and come crawling out of the woodwork to beat you up and steal your lunch money. You need to dissuade people who are going to try to exploit you first: if someone betrays you, you need to retaliate.

To explain his case, Axelrod starts with a classic game theory experiment: the Prisoner’s dilemma.

II.

A brief explanation from Wikipedia:

The prisoner's dilemma is a game theory thought experiment involving two rational agents, each of whom can either cooperate for mutual benefit or betray their partner ("defect") for individual gain. The dilemma arises from the fact that while defecting is rational for each agent, cooperation yields a higher payoff for each.

For rational players in a Prisoner’s Dilemma, the smartest option is always to defect. The only stable equilibrium is for both players to do so: the worst of all worlds.

But while considering the problem, Axelrod came to a foundational insight that would change the game forever. He realized that defection is only optimal iff:

You are playing against a single rational opponent, AND
Know the precise number of games you will be playing.

If you play an uncertain number of games all against the same opponent—known as an Iterated Prisoner’s Dilemma—defection is no longer guaranteed to be the optimal strategy. The longer the game and more responsive the opponent, the more viable cooperation-based strategies become.

To test this theory, Axelrod held two iterated prisoner’s dilemma tournaments, calling on “expert strategists from political science, sociology, economics, psychology, and mathematics” to submit their strategies and see which came out on top. It turns out that the optimal strategy was the simplest strategy submitted, named TIT FOR TAT. TIT FOR TAT’s entire strategy can be operationalized by just two statements:

Cooperate.
Match my opponent’s last move.

The book goes into more detail about the math and game theory behind TIT FOR TAT (and the tournament in general), but the basic appeals of TIT FOR TAT are astoundingly simple. TIT FOR TAT’s major strengths are:

Being nice. By never defecting first, it never makes enemies out of anyone unless it really has to. If two nice strategies meet each other, they will cooperate until the game ends: over the course of a tournament, these gains really add up.
Being retaliatory. By always returning a defection for a defection, TIT FOR TAT avoids being exploited too much by particularly nasty strategies. Being retaliatory is the key feature that allows TIT FOR TAT to dominate in a cold world full of evil: fool me three times, and well, you know.
Being forgiving. By being willing to return to cooperation after a defection—so long as its opponent cooperates first and eats a defection for it—TIT FOR TAT avoids needlessly punishing strategies that have defected in the past but might be willing to cooperate in the future. TIT FOR TAT can still get stuck in revenge spirals, but this problem can be mitigated e.g. returning 9/10ths of a TIT for a TAT instead of being equally retaliatory.

Interestingly, this means that TIT FOR TAT literally cannot outscore its opponent in any individual game. Instead of succumbing to tunnel vision and trying to win every game it plays, TIT FOR TAT does pretty well across the whole gamut of strategies. In Iterated Prisoner’s Dilemma tournaments, being brilliant is overrated: better to be dependable instead.

III.

The rest of the book is dedicated to proving to expanding upon these core insights in fascinating ways. Axelrod does come off as if he’s desperately trying to convince you that his framework solves all of your problems, but it was a real delight to see how widely applicable it actually was; apparently TIT FOR TAT and its variants can be found everywhere from trench warfare to bacterial evolution. These parts of the book are light on theory and heavy on anecdotes, making for fun reading.

(My favorite case study from the book was about bacterial evolution. It turns out that cooperation is a winning evolutionary strategy in all organisms that have repeated interactions even if they don’t have brains. And once the first eukaryotes developed, every cell in these eukaryotes suddenly developed an insanely strong incentive to cooperate, and kept evolving in that direction until we ended up with us! The trench warfare section is also really crazy—the emergence of cooperation itself wasn’t very surprising, but the degree to which that cooperation conformed to Axelrod’s suggestions, particularly TIT FOR TAT, is quite something.)

Finally, we get to Axelrod’s life advice, which is also very simple:

Don’t be envious.
Don’t be the first to defect.
Reciprocate both cooperation and defection.
Don’t be too clever.

The first three should be pretty self-explanatory, since they’re just the design principles behind TIT FOR TAT. TIT FOR TAT will never try to outscore its opponent; TIT FOR TAT will never defect first; TIT FOR TAT will never let a good deed go unrewarded (nor a bad deed unpunished). But the fourth is perhaps the most interesting, and deserves some further explanation.

See, I omitted a key detail in my earlier explanation of the two tournaments. They weren’t held simultaneously: they were held sequentially. In other words,

Players competed in the first tournament.
Axelrod analyzed the results and published his findings
Players—now armed with the ability to make complex and devious strategies that would have dominated the first tournament—competed again.

The funny thing is, many of the strategies in the second tournament were too smart by half. In particular, the meanest strategies choked themselves out: in the beginning stages of the tournament, they did TOO well against overly forgiving strategies, and knocked them out of the competition. Their remaining opponents were either nice but retaliatory, like TIT FOR TAT, or were mean and exploitative themselves—and mean strategies do not play nice with each other. By knocking out all of the softies, they ruined the very environment that allowed them to thrive in the first place. Talk about suffering from your own success.

Meanwhile, TIT FOR TAT was retaliatory enough to survive playing against the meaner strategies and played perfectly with the nice strategies. So it won. Again.

Combined, these two insights explain what was missing from my original intuition. When trying to explain cooperation to people, I had neglected the importance of retaliation and simplicity. Being nice doesn’t mean being a pushover: it just means extending cooperation to unfamiliar strangers. Unfortunately, Axelrod does not advocate turning the other cheek. (Sorry, Jesus.)

Furthermore, in real life, you’re not playing against fixed opponents: you choose who to play with. If you cooperate with nice people and retaliate against mean people, strangers will figure out your gameplan very quickly; you’ll begin to attract other retaliatory cooperators AND weed out backstabbers. That’s a win-win!

IV.

The last part of the book—possibly the most important part—is about how to design systems that promote cooperation. Axelrod gives three basic pieces of advice for reformers:

Make the future more important. As we saw earlier, the more interactions you expect to have with someone, the more important it is to cultivate trust and respect. On the other hand, completely anonymous and randomized interactions are ripe for betrayal, since you can’t ever build relationships with people: these kinds of systems tend to be nasty (see: 4chan). So if you want your system to promote cooperation, you should at minimum include some kind of identification system—even just a username—and may want to consider making a player’s action history publicly available.
Change the payoffs. The more you gain from cooperation, the more people will want to cooperate with each other. The more you gain from defection, the more people will defect. Thankfully, tasks in the real world are raid bosses: most things worth doing are too hard to do alone.
Teaching players about cooperation theory. If people know that TIT FOR TAT is the best strategy, they’re more likely to adopt it themselves, and then you can spam cooperation with them. It’s not a zero-sum gain: everyone wins!

I found this section particularly interesting, because its insights are reflected in platform design across the internet. I pointed out 4chan earlier because it’s nasty by design: when identities are anonymized and only persist within message boards, chaos is sure to follow. In contrast, Reddit is mostly anonymous, but their choice to use consistent usernames across all message boards and allow you to look at someone’s post history (and Reddit karma) in order to determine whether they’re a good-faith actor in advance. So, if you ever want to run a club or host a recurring group meetup of any kind, you’d do well to take The Axe’s advice.

I know there have been huge advancements in game theory since Axelrod published his book (duh). I'll probably read about them eventually, and maybe even write about them too. But this book was just so simple and fun to read because it is a generalist book, not a specialist book. You don't need any math background beyond high school algebra to understand what’s going on, and most of the people you meet IRL aren't game theory wizards anyway. Hell, it’s not like I'm a game theory wizard— I haven’t gotten into the weeds of game theory and probably never will—so this book was the 20% that reaps 80% of the rewards for me, along with confirming all of my pre-existing pro-cooperation biases.

TIT FOR TAT is kindergarten playground stuff. The core intuitions are just so simple: be nice, be retaliatory and be forgiving. It’s that easy!

V.

Unfortunately, most real-life social interactions don’t cleanly map onto the pure iterated Prisoner’s Dilemma. IRL, we also have to consider complicating factors like:

Different types of interactions. Many interactions are one-offs with complete strangers. Even if you do learn to recognize these people, you don’t always have the ability to punish their defections—you can’t just go around beating up people who litter in parks. Other types of interactions, like those with multiple players, are simply too complex for simple strategies like TIT FOR TAT.

Different types of environments. Unlike the tournament participants, you are not beholden to a single strategy. You can get harsher in cutthroat environments, nicer in more cooperative ones, or potentially vice versa depending on how you want to play. In the book, Axelrod comments that TIT FOR TAT wasn’t actually the optimal strategy for the first tournament; in fact, had TIT FOR TWO TATS been entered, it would have won outright, and only lost in the second tournament because the average strategy had become meaner!

Messy signals. Real life is not restricted to the simple “cooperation OR defection” framework used in the Prisoner’s Dilemma. Instead, IRL actions exist on some weird multi-dimensional framework of axes like friendliness, manipulativeness, genuine altruism, self-interested calculation, etc. And don’t even get me started on dealing with ambiguous and hard-to-interpret signals.

Inconsistent strategies. I dunno about you, but I don’t particularly like the idea of becoming some cold and calculating game theory wizard—I kinda like acting in accordance with my values. Without the guarantee that your game partners will act rationally (let alone yourself, for that matter), it’s risky to blindly follow game-theoretically optimal strategies in IRL interactions, particularly when the strategy in question is prone to:

Death spirals. TIT FOR TAT advocates punishing every defection immediately and with equal force. This is all well and good in games that lack ambiguity. In real life, it’s possible to punish cooperation by accident, whether due to misinterpreted signals or even just unfortunate mishaps (“trust me bro my finger slipped, I didn’t mean it”). If your opponent is also playing by TIT FOR TAT, they’ll then retaliate against your actual defection—which will cause you to retaliate in turn—ergo the endless death spiral.

To solve this issue, Axelrod suggests a strategy called nine-tenths TIT FOR TAT, which chooses to forgive every tenth defection in order to end death spirals. This might be optimal in a tournament setting, but IRL, you actually can just turn the other cheek and eat a loss sometimes; TIT FOR TAT doesn’t do gifts or favors, but I do, because I know my friends will help me out when I need it in turn.

However, I think there’s an even better method of dealing with defection:

Choose your dance partner(s). TIT FOR TAT was developed in a game that chains players to their opponents and forces them to play together. But if you can pick and choose your friends, you don’t have to try and change the behavior of your bad friends. If your starting environment sucks, go find a better one!

VI.

Considering all of that, it would probably be a mistake to naively apply TIT FOR TAT in real life. Sorry gang: looks like socializing isn’t solved yet.

That being said, the book did change my day-to-day life in three key ways:

It relieved my sense of pessimism about whether cooperation is secretly stupid and naive.
It curbed my tendency to engage in dunks or takedowns, particularly when I am uncertain about whether someone is acting in good or bad faith: these days I try to just disengage.
It made me feel much better about retaliating in response to explicit and unambiguous defection. Having been raised Catholic, this was a pretty important principle that I hadn’t quite internalized before.

Was I kind of already on the path to making these changes? I mean, probably. But it's nice to have the principles laid out so explicitly, even if you already expect to agree with Axelrod—the finer points are interesting in their own right.

And also, the book was damn fun to read!