113 | Cailin O'Connor on Game Theory, Evolution, and the Origins of Unfairness
What this covers
Cailin O'Connor builds a case that social inequality emerges predictably from basic strategic interaction, using evolutionary game theory as the engine. The conversation moves through how the same formal machinery—payoffs, strategy choices, population dynamics—models three distinct processes: evolution by natural selection, cultural evolution, and individual learning. O'Connor then traces how game theory illuminates behavior across biology and human society, from costly animal signals to labor markets, before arriving at her central claim: once populations split into identifiable groups, bargaining games unlock a whole class of unfair equilibria that cannot exist otherwise. The result is that inequality and convention lock together, reproducing themselves through rational self-interest alone—no malice required.
The substance spans from foundational game theory concepts like Nash Equilibrium and the prisoner's dilemma through signalling games and their twin logic of conflict of interest signalling, then into how [spontaneous symmetry breaking](concept:Spontaneous Symmetry Breaking) makes arbitrary advantages sticky. O'Connor walks through gender and racial inequality as products of the same dynamics: labor division establishes who controls resources, which determines bargaining power, which becomes encoded in what counts as fair. She addresses the hard part—that genuine fairness to disadvantaged groups requires active disruption, not passive acceptance, because the advantaged rationally resist conceding what they have. The conversation also treats how norms erode in belief before behavior shifts, why the prisoner's dilemma puzzles us as a window on altruism, and how model-building disciplines speculative stories by forcing rigor onto evolutionary claims.
O'Connor argues that social inequality between identity groups emerges naturally from evolutionary game-theoretic dynamics—via symmetry-breaking once identity markers exist—even without malice, which means inequity is the cross-cultural rule rather than the exception and requires active rather than passive correction.
- Bargaining games have multiple equilibria, including unfair ones, that become reachable only once a population is split into identifiable groups
- Disadvantaged group members behave rationally given their social situation, so unfairness arises without anyone being evil
- Because these forces are basic to learning and strategic interaction, inequity recurs and must be continually counteracted
Game theory models strategic interactions using players, strategies, payoffs, and information to predict and explain behavior.
- Game theory is a branch of mathematics for analyzing strategic interactions—situations with multiple actors (humans, animals, plants, bacteria) who have interests and who care what the other actors do—by building simplified mathematical models called games and analyzing them to predict or explain behavior.
“it's a branch of mathematics where the goal is to look at strategic interactions... an interaction where you have multiple actors who have some kind of interest, so these could be humans, but they also could be animals or even something like trees or bacteria”
- A game in game theory is defined by four elements: the players (who is in the strategic scenario), their possible strategies (what actions they can take), their payoffs (the benefits or detriments resulting from combinations of strategies), and information (what the players know about the setup and about the other players).
“you can define it using four elements. So the first thing is the players... The second thing is their possible strategies... The third thing is their pay-offs... And then information is the last thing that helps define a game.”
- Game theorists do not assume rewards can be exactly quantified; rather, by assigning specific numbers to represent payoffs, they make interactions precise enough to analyze, accepting that this is a good-enough approximation rather than a true measurement.
“the idea is we can use specific numbers to represent those rewards, and then by making them precise, we can get a kind of good enough or approximate representation that we can also analyse and better understand”
- Coordination games are scenarios with common interest where players get shared payoffs by coordinating their behavior, but multiple ways to coordinate exist so the interesting question is how actors settle on one; bargaining games are a subclass concerning resource division (money, time, effort), which is ubiquitous in human life and requires coordination.
“these are scenarios where there's common interest between the players... they're gonna get shared payoffs. And in coordination games, the way they get shared payoffs is by somehow coordinating behaviour. But then the kind of thing that makes the game interesting, is that there are multiple ways to coordinate behaviour”
Inequitable social conventions lock in through bargaining dynamics even when disadvantaged actors behave rationally within their position.
- The argument that bargaining across a society should always yield fair outcomes—because nobody accepts an unfair amount and so refuses to play—is a poor analysis, because people make rational choices given the social situation they are actually in, which can lock in inequitable conventions even when each disadvantaged actor is behaving sensibly.
“if we look at bargaining games and bargaining in a whole society, we should always expect to end up at a fair outcome... Now, I think that analysis ends up being a terrible kind of analysis because people are rational given the social situation they're in”
- Splitting a bargaining population into two identity groups (men/women, races) introduces an asymmetry: identity markers let actors condition strategies on group membership ('type A treats type B this way'), which unlocks a whole class of inequitable equilibria that cannot evolve without groups, and these inequitable conventions pop out of the dynamics robustly regardless of model details—even though, in the basic model, which group ends up advantaged is an accident of history.
“As soon as you have two groups, you have these identity markers where you can say, "Well, I'm type A. You're type B and type As treat type Bs in this kind of way." And so that allows inequality to emerge”
- Because disadvantaged outcomes arise from people behaving sensibly within their social positions, the analysis suggests that, for some kinds of discriminatory behavior, one might counterfactually have behaved the same way in the other group's position (e.g., if socialized as a man, one would have learned to expect more), providing a basis for extending understanding to people who behave in discriminatory ways—though this does not lessen the moral wrongness of discrimination.
“If I had been raised as a man, I think I very reasonably would have learned... to expect to get more than I'm socialized to expect to get as a woman... it maybe gives some locus for extending understanding to people who are behaving in discriminatory ways”
- Inequity has a life of its own: if you have social identity groups and everyone tries to learn what is best for themselves, inequity can emerge purely from those non-offensive preconditions, which explains why inequity based on social identity is the cross-cultural rule rather than the exception—and implies it requires active rather than passive responses, since basic forces of learning and strategic interaction continually push toward inequitable patterns.
“As soon as you have two groups, you have these identity markers where you can say, "Well, I'm type A. You're type B and type As treat type Bs in this kind of way." And so that allows inequality to emerge”
Altruism emerges from repeated interactions that reward cooperation and from kin selection favoring genes helping relatives.
- Altruism that seems individually irrational is explained by two mechanisms: repeated interactions change the game's structure so cooperating now can be rational because it elicits future cooperation, and kin selection means genes for altruism are favored because altruists directed toward kin are disproportionately helping other altruists who share those genes.
“one is the one you just brought up, that we're in repeated interactions with others... it can make it rational now to cooperate on the assumption that cooperating now will get them to cooperate with you later”
- From the gene's eye view popularized by Richard Dawkins, altruism toward kin makes complete sense because an altruistic gene benefits other copies of itself residing in relatives, even though from the individual's perspective helping kin reproduce at one's own expense looks irrational.
“this was an idea largely popularized by Richard Dawkins from a gene's eye view, if I'm a little altruistic gene, I would like to benefit other genes just like me”
- The deep reason the prisoner's dilemma is so fascinating is that it is essentially an analysis of altruism: cooperating lowers your own payoff while raising others', which is materially altruistic, yet the game's structure says altruism is not individually rational—so the puzzle is why we observe so much altruism in humans and animals.
“in the end, it's an analysis of altruism, so if you cooperate in the prisoner's dilemma, you've done something that will lower your payoff, but increase the pay off of the other players, so you've done something kind of materially, inherently altruistic”
The prisoner's dilemma shows individually rational choices produce socially worse outcomes and explains why cooperation is hard.
- In a basic population without added complexity, the prisoner's dilemma's Nash equilibrium of mutual defection is also the evolutionarily stable strategy: the population will evolve to all defectors, because cooperators are exploited and defectors have more offspring, so any cooperative cluster is undermined by invading defectors.
“the prisoner's dilemma in a totally basic population is also the evolutionarily stable strategy. So if you have a population without adding extra bells and whistles, it will just evolve to always all defectors”
- In the prisoner's dilemma, the payoff structure makes defection individually rational—the best outcome is to defect while the other cooperates—yet if both defect they are worse off than if both had cooperated, creating a dilemma where individually rational choices produce a socially worse outcome.
“there's a payoff structure that incentivizes you to defect, but creates a situation where if you both defected, you would prefer that you had both cooperated, so that's the dilemma of the game that it's individually rational to choose to defect, but from a kind of social level you can all do better by not all defecting”
- Taking a meta step to escape an inequitable equilibrium is more plausible in bargaining than in the prisoner's dilemma: in the prisoner's dilemma even allowing communication does not guarantee cooperation (the only equilibrium is bad), whereas the bargaining game has a fair equilibrium available—but conversation alone may still fail because those getting more benefit from getting more and often engage in special reasoning to justify the status quo that benefits them.
“in studies of the prisoner's dilemma like people being able to talk and do something beneficial for them, it does not guarantee that people don't defect. With bargaining, it's different... the fair outcome is an equilibrium”
Game-theoretic modeling has limits and yields different insights depending on use: direct prediction, pattern explanation, possibility proof, reasoning aid.
- The Lenski long-term evolution experiment, in which a subset of bacteria evolved to eat a less efficient but less-contested sugar, is not best analyzed with a game because a game models an interaction where organisms care what the other does; instead a model about utilizing underused resources is more appropriate—illustrating the limits of the game-theoretic paradigm.
“I don't think that would probably be best handled using a game... This probably, you'd wanna use a model that was less game theoretic and more about utilizing underused resources”
- What a simplified game model can tell you about the real world depends entirely on what you do with it: some models are compared directly to data sets of human behavior, some are used to explain broad empirical patterns like cross-cultural regularities, some are proof-of-possibility against impossibility claims, and some serve as aids to organize reasoning rather than to do detailed analysis.
“A lot of people will look at simplified models like games... and say, "Well, this is so simple, what can you even do with this?"... I think the answer totally has to do with, what are you doing with it?”
- Evolutionary game theory and modeling help defeat 'just-so stories' in biology by forcing speculative narratives (e.g., that senescence exists so younger birds can have more food) to be replaced with rigorous analysis—putting numbers on benefits and translating them into offspring—thereby constraining which evolutionary narratives are defensible.
“evolutionarily modeling, including evolutionary Game Theory helped throw out a lot of Just So Stories in biology and provide at least some sort of way to constrain what sorts of stories or narratives people could tell about the evolution of certain traits”
- One distinct use of game-theoretic modeling is proof-of-possibility: against Quine's natural-language skepticism that language cannot evolve without prior language to establish conventions, modeling work by David Lewis and Brian Skyrms showed that extremely simple agents can learn to coordinate on a signal meaning something without any prior language.
“Quine was someone in this tradition who argued you can't evolve language naturally... But modeling work by David Lewis and then Brian Skyrms showed that, well no, you can. You can have extremely simple agents learn how to coordinate on a signal to mean something without needing any sort of prior language”
- Modeling is useful because it lets you build precise structures you can analyze rigorously, but the inherent downside is that you may lose the precision of the real world or misrepresent it significantly, so there are always deep questions about whether a model is the right model and whether its conclusions hold in a more complex reality.
“the downside to that is that you're gonna lose some of the precision of the real world or you might represent it incorrectly in some really significant way, and so there are always gonna be these deep questions about, "Is your model the right model?"”
Norms are stable through payoff forces but erode through dissatisfaction; real change requires disruptive action, not voluntary concession.
- Norms and conventions are very stable because if everyone adheres, payoff forces push everyone to keep adhering, but a norm can be eroded—people can become dissatisfied with it and come to believe it unjust—well before any behavior actually changes; real change requires someone to actually act differently (the disadvantaged making higher demands and disrupting things, or the advantaged conceding more), and the disruptive route is observed more often than voluntary concession.
“you can have situations where people stop liking a norm before they stop adhering to it. Everyone's still doing it but you can have people becoming dissatisfied with it, learning more about it, coming to believe that it's injust”
- The claim that fairness benefits everyone is too simple: while movements like feminism can free people from constrained gender roles (a real benefit to the advantaged group too), finite resources distributed inequitably mean that moving toward fairness genuinely asks advantaged groups to give up some of what they have, which is why civil rights, Black Lives Matter, and feminist movements meet so much resistance—people aren't dumb and know fairness costs them.
“it's really too simple to say fairness benefits all of us and it ignores a lot of the reasons why movements towards fairness like civil rights movement or Black Lives Matter movement or feminist movements meet so much resistance because people aren't dumb”
Gender initially divides labor; once established, labor conventions determine resource distribution and power imbalances between groups.
- The first and most important role gender plays in many societies is as a locus for dividing labor; in traditional societies division of labor was necessary because many skilled tasks were hard to learn, so conventions assigning tasks by gender (women make rope, men fire pottery) benefited groups—and once gender determined who does what, it also came to determine who gets how much and who 'deserves' what, with those controlling food production positioned to demand more.
“The first most important role that gender plays in many societies is as a locus for dividing labour... it was completely necessary to divide labour 'cause there were so many skilled tasks that required doing that were very hard to learn to do”
- Gender egalitarianism varies cross-culturally: some cultures have norms with little power imbalance and similar labor burdens between men and women, while others are much less egalitarian, showing these are conventions emerging partly from chance and historical accident rather than inevitable outcomes—and demonstrating that groups can become more fair.
“there are cultures that have pretty gender egalitarian norms and conventions where there isn't a big power imbalance between men and women... and then there are cultures that are much less gender egalitarian”
Evolutionary game theory applies across genetic, cultural, and individual learning using the same formal machinery with different assumptions.
- Evolutionary game-theoretic models represent three distinct processes under different assumptions: evolution by natural selection (payoffs translate into more offspring), cultural evolution (successful behaviors are imitated and transmitted), and individual learning within a lifetime (an agent learns which strategy does best in its environment and sticks with it), so the same formal machinery models genetic, cultural, and learned dynamics.
“some models are really thinking about evolution by natural selection... the pay-off they get is some kind of benefit that allows them to have more offspring on average”
- Evolutionary game theory, developed largely by John Maynard Smith, drops the assumption that players rationally deliberate over strategies; instead it takes a population of agents with given behaviors and asks how the population evolves, because most organisms engaged in strategic interactions (bargaining, signalling, altruism) have evolved their behaviors rather than reasoning them out.
“biologists thought, "Well, we have animals, organisms of all sorts, engaged in strategic interactions all across the biological world... but most of them, we know, aren't sitting down and engaging in a rational deliberation." Instead, a lot of their behaviours have evolved”
Nash equilibria predict behavior because actors have no incentive to deviate, confirmed by laboratory experiments showing play near equilibrium.
- A Nash equilibrium is a set of strategies where, when actors play them, none of the actors has any incentive to switch to a different strategy; it is important because it represents an approximately stable set of behaviors and therefore predicts what actors will do, and laboratory experiments show people often play Nash equilibria or something like them.
“you're looking for sets of strategies where, when actors are playing them, none of the actors wanna change and do something else. There is no incentive for them to switch strategies. And those sets of strategies are Nash equilibria”
- Choosing which side of the road to drive on is a coordination game where matching strategies (both right or both left) yields good payoffs and mismatching causes crashes; which side gets chosen is arbitrary—a case of spontaneous symmetry breaking—but coordinating on the same side is what matters, explaining cross-country diversity with within-country regularity.
“let's say these represent driving on the right side of the road and the left side of the road. And you get payoffs if you both pick A... But if one of you picks A and one picks B, you don't get good payoffs because you crashed into each other”
Signaling theory explains costly traits: common-interest signaling coordinates ubiquitously; conflict-of-interest signaling uses cost to guarantee honesty.
- Conflict-of-interest signalling theory, using cost to guarantee honesty, explains extravagant biological adaptations like the peacock's tail or the turkey's face: such costly traits are reasonable adaptations if you count the social/reproductive environment as a legitimate part of an organism's environment, because expending high costs to signal quality yields social or reproductive benefits.
“if you count your social environment as a legitimate part of your environment, then it's totally reasonable to expend a lot of costs to signal certain things about you that are gonna get you social benefits, or in the case of the peacock, reproductive benefits”
- Game-theoretic signalling research divides into two camps: common-interest signalling, where sender and receiver want the same thing (and which explains why signalling is ubiquitous, down to bacteria and plants, because coordination requires transmitting information), and conflict-of-interest signalling, where sender and receiver interests may diverge (as in job-market hiring where every candidate wants to be hired regardless of quality).
“usually you wanna break up this literature on Game Theory and signalling approximately into two big camps, and one is common interest signalling, signals when the two individuals involved want the same thing”
Natural outcomes carry no moral weight; the naturalistic fallacy must be resisted even when inequity emerges from ordinary dynamics.
- When the analysis says inequity emerges 'naturally,' this means only that it is the outcome that arises naturally from the dynamics—it must not be given moral weight; the naturalistic fallacy of treating what is natural as good or as unavoidable should be explicitly resisted.
“when we say "natural", we'd say something like, "It is the thing that emerges naturally" but we certainly wouldn't wanna give it the moral weight”
- While the bare model assigns advantage by random chance, more realistic cases involve specific factors that bias which group ends up advantaged: physical sex differences serve as a symmetry-breaker for gender outcomes, and for racial or cultural groups, factors like power, economic advantage, and minority status determine who gets more in conventional bargaining outcomes.
“If we're thinking about gender facts, about physical sex differences are sort of an important symmetry-breaker in determining who's gonna end up with better outcomes. If we think about cultural groups or racial groups... there are gonna be facts having to do with power, economic advantage, maybe with minority status”
Mixed strategies in games have population analogs: either populations split into distinct strategy players or individuals evolve stochastic behavior.
- Mixed strategies in individual play (randomizing among actions, as is optimal even in complete-information games like rock-paper-scissors) have a population analog: either part of the population fixedly plays one strategy and another part plays another, creating variation across the population, or individuals each evolve stochastic behavior, as with escape behavior where every prey animal randomizes its direction rather than splitting into 'left-runners' and 'right-runners.'
“if you look at rock paper scissors, even though everyone knows exactly everything about how that game works, your best thing is to randomly mix among your strategies”
- In a bargaining game where players can demand a third, half, or two-thirds of a resource (with overdemand yielding poor payoffs), the equal-split outcome is a special symmetric equilibrium where both players do the exact same thing and perfectly divide the resource; philosophers like Jason Alexander and Brian Skyrms and economists like Peyton Young have used this to explain why norms of justice and fairness exist, while unequal divisions (e.g., one-third/two-thirds, or even 99/1) are equally valid equilibria.
“the equal split outcomes are special because you and I can do the exact same thing and perfectly divide our resource. So it's a kind of special equilibria where you're being fair to each other. And other philosophers like Jason Alexander and Brian Skyrms and economists like Peyton Young have used this to try to explain why we have norms for justice and fairness”
- An evolutionarily stable strategy is a solution concept for evolving populations: a strategy is evolutionarily stable if, when the whole population plays it, it resists invasion by mutant strategies—mutants die off because the dominant strategy does better against itself (or against the mutants) than the mutants do, so any variant entering the population fails to spread.
“What are the strategies, where if the whole population is playing that strategy, it's stable to invasion by other strategies?... If we introduced a little mutant who was doing something else, would this population still be stable? Would that mutant die-off?”
Group selection probably does not explain altruism primarily, but this does not justify generalizing that group selection is unimportant for other traits.
- John Maynard Smith's game-theoretic models showed it is hard to get group selection for altruism to work, but the literature wrongly generalized this to conclude group selection is unimportant for everything; the correct conclusion is only that group selection probably is not the main explanation of altruism, not that it is unimportant for explaining other traits.
“John Maynard Smith did these very influential models, game theoretic models, where he showed that it's quite hard to get group selection of a certain type to work. And that's group selection for altruism”