Link to original article
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Coercion is an adaptation to scarcity; trust is an adaptation to abundance, published by Richard Ngo on May 23, 2023 on LessWrong.
There’s a sense in which I’m postulating a trillion dollar bill lying on the ground. If the skills I’ve talked about so far in this sequence would lead to so much more flourishing, why haven’t they become far more common? I’ve already given a partial answer: that our evolutionary environment was much more dangerous than our current environment. But I want to extend this to a more general answer: that coercion is an adaptation to scarcity; and that we only very recently left the era in which scarcity was the dominant feature of people’s lives.
Under conditions of scarcity, you don’t have enough slack that you can afford to take risks. If misbehavior from any individual in the group would risk the lives of many others, you have to coerce them into staying in line; if a loss on any gamble would leave you ruined, then you need to avoid taking those gambles even when they’re winning in expectation. (Poker players think a lot about this when trying to manage their bankroll—if a game has high-enough stakes that they’d be out of money if they lost, they have to avoid it, even if they expect to make money there on average.)
By contrast, in an abundant environment, you can take the optimal long-term strategy, even if there’s a risk that it’ll leave you way down in the short term. In particular, you can put effort into building trust with others, even though that leaves you more vulnerable to being let down or betrayed. With that trust, you can receive a huge range of gains from cooperation.
Western societies are incredibly abundant in many ways. As a citizen, you face almost zero risk of starvation, dying in a war, or exile from your country; meanwhile deaths from most diseases and accidents are dramatically lower than in the past. There’s more career flexibility than there ever has been before: there are many routes to success, including self-employment. And society is far richer than it’s ever been before: the median person in a western society is incredibly wealthy both by historical standards and by the standards of most people across the world. Even welfare recipients are still well off by historical and global standards.
What does it look like to internalize a feeling of abundance? It might involve not forcing yourself to study when you don’t feel like it; or being less defensive when people criticize you; or deciding to be less harsh on yourself about the possibility that you might fail. The more slack you have, the more of a risk you can take by doing this—in each case it’s a gamble, but one which more often than not pays off in the long run by reducing internal conflict and allowing you to act more coherently towards your goals. Similarly, excitement about the future is a gamble: if you get excited about something which doesn’t work out, you’ll be left in a worse position than if your expectations had stayed low the whole time. The way that rich people can get higher returns to investment because they can absorb more risk is analogous to how fear forces us to take suboptimal actions.
What if you think, like I do, that we live at the hinge of history, and our actions could have major effects on the far future—and in particular that there’s a significant possibility of existential risk from AGI? I agree that this puts us in more of a position of scarcity and danger than we otherwise would be (although I disagree with those who have very high credence in catastrophe). But the more complex the problems we face, the more counterproductive scarcity mindset is. In particular, AGI safety requires creative paradigm-shifting research, and large-scale coordination; those are both hard to achieve from a scarcity mindset. In other words, coercion at a ...
view more