The Deutsch Files II

Brett Hall and I interview David Deutsch, physicist and author of The Beginning of Infinity. Also see The Deutsch Files I.

The universality of computation and explanation

Naval: So let’s go through The Fabric of Reality—the four theories. Feel free to start wherever you’d like, but the four theories that you think comprise the Theory of Everything and maybe especially what are the biggest things that even peers, colleagues, contemporaries don’t understand or don’t fully appreciate that makes each one of these deeper or perhaps more counterintuitive or more interesting than it might be at first glance.

David Deutsch: Well, I don’t know, we can start with computers. As I said in the book, it’s hard actually to speak about any one of those things without mentioning the other three. But if we start with computers, I think there’s something really fundamental that Turing discovered, or rediscovered, because I think that Babbage and Lovelace also understood it, more or less. That’s the universality of computation, that computation is physically universal. So there are several ways of putting this, like a computer can mimic any physical object, or a computer can perform the computations that any other computer can perform. Now putting it the second way, it sounds like it’s a statement about all kinds of different computers and has nothing to do with trees and garages and windows and so on, but actually it has to do with everything. And therefore, people still, even today, are saying things like, “How do we know the brain is a computer? You’re just assuming the brain is a computer. Like in the 19th century, people thought the brain was a steam engine.” And I think Searle is one of the people who says that kind of thing, or has said that kind of thing.

In order to understand Turing’s discovery, you’ve got to understand several things about it. One of which is that it’s a theory of physics and that is denied almost wholesale by mathematicians. So mathematicians are used to the theory of computation being a branch of mathematics. They love the theorems that you can prove and the theorems that you can’t prove and so on. And it’s not that they don’t want to admit, it’s not quite it. It’s that learning to be a mathematician apparently means adopting a certain worldview that makes it very hard to understand that computation is a physical process and is governed by laws of physics, which could be different. Whereas the laws of logic, they think, couldn’t be different, and therefore that things like whether P equals NP and whether the brain is a computer and so on isn’t a matter of physics.

But it is, and the best physics we know (which could be wrong), says that computers are universal, that in a certain sense Turing’s computers are universal, and in a certain sense quantum computers are universal, or will be when they’re built, if they’re built.

Brett Hall: Just on this, how do we know the brain is a computer? Turing’s thesis would say that all physical processes can be computed. So what a tree is doing, we can write a program and a Turing machine would be able to capture that. But the tree is not a computer, but the brain is?

David: Well, the tree isn’t a general purpose computer. But you can think of Turing’s thesis, whatever you call it, the other way around as well, because the reason he wanted to make this imaginary machine out of paper is not that he wanted to understand paper, or as Feynman said he should have, but that he wanted to have a model of computation and he wanted to be able to say, to conjecture that it is obvious that anything that can be computed can be computed by this paper.

Now, that means that he’s also assuming that this paper can also compute whatever a tree can compute, because you could regard the tree as a computer, and then Turing’s saying whatever it can compute is a subset of what Turing machines can compute and Turing machines are the ultimate. There’s nothing beyond that. That’s another thing that’s really important that when it comes to universality of explanation, people don’t get it because they don’t even get the universality of computation. They don’t understand that if someone says, “What if the aliens come from Alpha Centauri and they have better computers than us?” It’s impossible. They can have computers that are faster and have more memory, but that’s it. Our computers are the limit of what can be computed by anything in the universe. Unless quantum theory is wrong and so on, but that’s not what they’re saying. They’re not saying maybe quantum theory is wrong.

They’re failing to make the connection in Turing’s argument between physical objects and mathematical objects—imaginary paper and anything else, like trees. I don’t know if that’s the thing that people tend to get wrong, but that’s one thing that I’ve seen a lot lately.

The growth of knowledge begins with problems

Naval: And that leads us into the others, both epistemology and quantum theory. Let’s go to the one where you, even though you may deny it, where I think you’ve made the most original contributions after computation, which is epistemology. We have to invoke his name just to point to it, but Popperian epistemology. What do people not appreciate or perhaps overlook or get wrong or not understand? Or is there another way of approaching it that might help people understand the fundamentals?

David: Yeah, it’s more of another way of approaching it. So people do credit Popper with certain things, but they are unimportant things by comparison with his actual philosophical discoveries. Like that scientific theory ought to be testable. That’s true, you know, or 99 percent true, or something like that, and it is reasonably important in order to distinguish things, as Popper wanted to do, to distinguish things like fundamental physics from Marxism. So it’s useful for that, but it’s not such a big deal. I think my colleague Matjaž Leonardis said last year that to him the most important concept in Popper is the concept of a problem. Once you’ve understood what Popper means by a problem, you have this other way of understanding what epistemology is and so on. And I think I’ve come around to agreeing with that, because all previous epistemologies assumed that knowledge is, well, sometimes it’s called justified true belief, but I think it’s wider than that.

I think the misconception is that we want knowledge because we want to rely on it, and therefore, wherever it comes from, which is mysterious, but that better be reliable too. That’s the intuitive idea that I think most people have and that most philosophers had over the millennia. Therefore, you want to say, well, what is absolutely certain? Is it the sayings of the gods, or of God? Is it immediate sensory perception? Or is it our tenuous memories of a previous life, like Plato tried to say? Because I think many serious philosophers have realized that the senses are imperfect and can be misleading. But then they said, okay, well, if not the senses, what can we rely on?

And then Immanuel Kant said, “Oh, pure reason, you rely on pure reason.” And that led him to all sorts of rather silly conclusions. Whereas if you have the idea of a problem, as Popper understood it, which usually in Popper, the word problem refers to good things, although there are bad problems as well, like the problems of suffering and so on. But he mainly uses problems as used in science as an interesting thing which we haven’t solved yet, which we haven’t understood yet.

And then, as soon as you think of science and rational thought generally as being about problems, then you lose the urge, the need to talk about where it comes from, because the problem is there to be solved, and the solution is what you want, not the justification of the solution by going back to first principles and proving that Jesus is the Son of God because he’s a descendant of King David because the prophecy said he had to be a descendant of King David. Therefore, we have to secretly invent a genealogy that goes right back to David in real life. And nobody has discovered a real genealogy that goes back that far, anywhere near that far.

And the truth of Christianity doesn’t depend on it. It’s the wrong way of thinking about Christianity. But then, when it comes to religions, people then, because of this epistemological error, because of this complete disregard of problems as the origin of the growth of knowledge, the more important the thing they want to say they know, the more they want to justify what they think is the origin of it.

So you have people waging wars and torturing each other to death because of their interpretation of what somebody who may not have existed said thousands of years ago and probably, even if he did say it, probably didn’t mean it in the way they mean today. We know because people do exactly this for people who lived a hundred years ago or indeed people who are alive today. So it’s a farce, but it’s also a tragedy, as Karl Marx said. The simple epistemological error leads to unlimited suffering. And it’s a common error, so I think if I had to pick something that most people don’t get about epistemology, it’s that the growth of knowledge begins with problems.

Problems are clashes between ideas

Brett: Let’s linger on that and focus a little bit more on the notion, therefore, of a problem in Popperian epistemology, given that most people will hear the word problem and think something negative. But you’ve already said that Popper speaks about problems as a good thing. Am I right in saying that it’s somehow a clash of ideas or where ideas are making claims about the one phenomena but are making different claims about or competing claims about this phenomena?

David: Yes. And not only about phenomena, about anything, you know, about morality and pure mathematics and you name it. So, yes, there can’t be any one definition of the concept problem and Popper doesn’t do definitions quite rightly. But I think thinking of a problem as a clash, and it’s got to be a clash of ideas, or interpretations, or theories, or so on, is illuminating.

Because if you think of it that way, then you start with the idea that they can’t both be true. I mean, that’s what the problem consists of, it’s realizing that they can’t both be true. It’s important to realize they could both be false, rather than say, we’ve got to find the true one. Usually they are both false, but usually there are important errors to correct, and usually there are important errors more in one of the clashing ideas than in the other.

Popper also stresses, and this is also quite important, speaking of clashes, that a clash of ideas is very beneficial even if they are never resolved. Even if the parties with the ideas never agree, because when the ideas come into conflict with each other, almost without the people knowing it or wanting it, they get changed. Because even if you come out of an argument saying, “Oh, I’ve really showed it to them!” right? What you mean is, you’ve thought of a new angle, which you didn’t have before going into the discussion. You’ve thought of a new angle on your own view, which makes you more sure of it than before. And, although it’s not good to be sure of things, this change, this way of changing, the confrontation between ideas as being beneficial because they cause change in the ideas is also a beneficial side effect of Popper’s concept of a problem.

Brett: I can hear the anti-Popperians or even the non-Popperians saying, but hold on, when you make an observation with a telescope of here’s Mercury, that’s an observation. That’s not an idea that you have. So the fact that it conflicts with the existing classical picture of how gravity works, that’s not a clash of ideas. It’s an observation that is clashing with grounds and an idea or a theory.

David: Yeah. Well, you had two theories at the time. Both of them had their adherents, general relativity and Newton’s theory. There were also other tangential ideas. Like if you believe Newton’s theory, or if you adopted Newton’s theory and you wanted to reconcile that with the observations, because the observations were also a theory, and you could say that the astronomers are wrong, which they did actually. People said about Eddington that his observations were wrong, and it was only actually in very recent times that it was uncontroversially discovered that Eddington’s observations were in fact right. Even though they were incredibly hard to do and he didn’t get enough credit all those decades. So you had theories about the observations.

Then there were theories to fix up Newton’s theory, like the theory that maybe there’s another planet that we don’t know about. And you could tie down that theory and say where the planet has to be, what its mass has to be, and so on. And you could then slowly rule out that this planet was there and Newton’s theory still be true. Of course, if Newton’s theory was wrong, then you can put the planet anywhere you like and make any modification you like to Newton’s theory. So that’s not how it works. We want good explanations.

At that time you’re speaking of, Newton’s theory was a good explanation. It had some problems with it. Einstein’s also. And all relevant experiments—the observations of Mercury, the observations of the Eclipse, and so on, and all the subsidiary observations as well, were all in conflict. And the argument improved those theories until the clash was such that Einstein’s theory was the only good explanation left. You had an infinite number of bad explanations, they’re always left. But the only good one was Einstein’s theory, and I know you were looking over there through your telescope at the non-Popperian, but I think he’s gone now.

Evolution is not the survival of the fittest

Naval: Let’s talk about one of the remaining two, quantum physics or evolution by natural selection.

David: There’s a very simple way in which people don’t get evolution, and it dates back to the old theories of evolution, like Lamarckism and I don’t know what they called Erasmus Darwin’s gradualism, or whatever, but anyway, these were attempts to account for the world around us without appealing to the supernatural. So both Lamarckism and Erasmus Darwin’s theory, I am probably not crediting the real author, the originator of that theory. They wanted to make sense of the world without appealing to the supernatural. And their ideas are still current, not under those names. Sometimes Lamarckism even under that name. And of course, there was Lysenkoism, which was a species of Lamarckism. But today, most scientifically minded people would say that they agree with Darwin’s theory of evolution, and then they would immediately, often, go on to say that after all, “the survival of the fittest”, and, you know, obviously the fittest are going to survive, and that’s not at all what Darwin’s theory says.

But, I actually think this battle is more important than the one between creationism and evolution. Because this battle, the battle between Lamarckism and Darwinism, or neo-Darwinism, whatever you want to call it. Again, we can’t pick a good name for it. This is about what is a scientific explanation, and creationism versus evolution is not about that. That’s about whether we want a scientific explanation. So, if somebody’s philosophy seeks a supernatural explanation of the world, then you can’t argue with that person about evolution. You’ve gotta argue with that person about that. That’s the philosophical argument. It has nothing to do with animals or evolution or anything like that. You have to engage with that idea of wanting the supernatural on completely different grounds and with different philosophical arguments to the ones that you would use about evolution. I think that’s more important. But you know. That’s just my opinion.

Naval: So you gave us the enticing tidbit that it’s not about survival of the fittest. What’s wrong with saying survival of the fittest?

David: Ah, well, it’s about the replication of genes. Or gene variants, if you want to be more precise still. It’s about the differential replication of gene variants, which is what gives it its connection with epistemology as well.

Naval: So it’s the survival of the best adapted gene? Or propagation of the best gene?Or genes encapsulate knowledge and it’s the growth of knowledge, and therefore the replication of the genes is a physical instantiation of that knowledge?

David: You can put it that way if you think of knowledge as information that has causal properties, but not everyone does. So there are theoretical biologists who try to develop numerical measures of fitness so that they can say genes evolve to maximize fitness. And fitness has got something to do with how many of your grandchildren survive and, you know, the very complicated mathematical thing. And Dawkins said, now I won’t be able to say it as well as he did, something like fitness is that quantity which appears to be maximized if what is actually maximized is the survival of genes. So, this is a very simple theory at one level. I said the other day that, in a sense, Darwin could have written his theory on one page, but it needed a book to explain it and he still hasn’t entirely succeeded. And the neo-Darwinians had to improve on it a little because he didn’t have a concept of gene because they hadn’t been invented. Or maybe they had with Mendel, but he didn’t really know that, even though they were contemporary.

Brett: One part of what I got from The Beginning of Infinity, which I didn’t even realize until I read the book, was that we understand far less about evolution by natural selection than most people think. Going through high school, you’re basically taught, if you encounter evolution by natural selection, well there’s the theory, it’s wrapped up in a nice little bundle, it’s almost like Newtonian physics, that explains everything about what’s going on in biological diversity. But you point out evolution by natural selection almost stands on equal footing, not quite, but there’s a mystery there at the heart, as there is at the mystery of the heart of what a person is and how a person creates knowledge. Can you illuminate that for people? What do you mean by we don’t really know everything about evolution?

David: Well, in both cases there’s a mystery. I think in the case of evolution that mystery is not as important to the foundations of the theory as the question of “What is a person?”, “What is knowledge?” has to do with artificial general intelligence. But in regard to evolution, it is a fact that we, despite having enormous amounts of computer power available, do not know how to make an artificial ecosystem as a simulation on a computer. What always happens is, when they try to make such a system, is that the functionality of the simulated organisms improves and improves and improves and then stops improving. And real evolution is nothing like that. Real evolution is going on all the time, changing, making new branches. There are new species evolving all the time and it’s just going faster and faster and there’s no end in sight to it. It’s open ended.

Brett: You talk about the robotic legs learning to walk in The Beginning of Infinity. Genuine biological evolution doesn’t have a goal in mind, but clearly with the graduate student who’s got these robotic legs that don’t yet walk, uses a so-called evolutionary algorithm such that by the end of a number of iterations it is walking. Well it’s been programmed with the goal of walking—

David: Yes, so I actually first realized this about evolution in a lecture that I was at about robots walking, that’s what I put in the book. In a way it was an amazing presentation. This was, I don’t know when it was, 1980s or something, maybe earlier. So computers weren’t as powerful in those days and these people were making actual robots, not simulations. So what I said just now is about simulations, but the same is true of robots as well. Robots have gotten much better now, but they still don’t do this thing that evolution does even slightly.

So I saw the videos that they had of the robot not walking very well and then walking better, and then walking in ways that they hadn’t foreseen. So they were saying, “Ah, this is creativity in evolution”. And so I thought, wow, you know, if I come back in a year or two, what will they be doing then? And then I saw. Oh, they won’t be doing anything new at all unless the graduate student thinks of it.

Naval: They don’t have their own problems. The problem is imposed from the outside. And so the problem it’s solving is just being solved from the outside. It’s just an instrument being used in the pursuit of solving that very specific, a very focused problem.

David: Yes. So another example I use to illustrate this, an example from physics, is that Newton’s theory of gravity had an arbitrary constant in it, which we now call capital G. I think historically it wasn’t G, it was mg—the mass of the earth times G or something, which was the fundamental constant. It’s neither here nor there, but it had a constant, which Newton didn’t know. And then later, Cavendish invented this very clever experiment to determine this constant. Now, I think Newton’s discovery was not incomplete by not knowing that constant.

His discovery was an explanation, and that explanation is the same before and after Cavendish. Cavendish, no doubt, used tremendous creativity to design the Cavendish apparatus and to make it measure G with an accuracy that you’d be amazed was possible in those days. He did that. That involved creativity, but that wasn’t creativity about gravity. That was creativity about brass balls, wires and so on—incredibly sophisticated. By the way, experimentation in science is hard.

I don’t know if that comes up anywhere in the four strands, but that’s another thing that people just don’t realize. They don’t realize that mistakes happen all the time. And to do an experiment where you can form a good explanation that you have measured the thing that you’re saying you have measured is very difficult. And sometimes beyond our technology or our knowledge at the moment. And so people just do a bad experiment and publish that. So that’s another thing that happens that people don’t get.

Brett: It was interesting when I asked you about this once, in terms of just making observations. Ostensibly what an experiment is, is here we’re making a precise observation. And the experimenter knows more than anyone about how the instrumentation works, and yet still errors can go wrong. Weirdly enough, that entire way of talking about experimentation comes to bear on a topic du jour, I suppose, of these UAPs, the UFOs, and that kind of thing.

You know, people thinking they’ve made this observation, and yet it’s not being done in a laboratory where it’s highly controlled and everyone understands it’s way worse than that. Here’s a thing that no one knows about, and yet we’re making grandiose claims about it.

David: That’s a very good example. And when challenged, people will always make a beeline for the authority. “Oh, you know, this was a USAF captain, you know, you’re impugning his status, you’re impugning his honesty” or whatever. Well, the real truth is that mistakes are everywhere, and everyone makes mistakes. And there’s no limit to the amount of mistakes we can make. And in scientific experimentation, almost all of the effort required to do a scientific experiment is forming theories about the errors, forming explanations of what errors there could be, and then forestalling them or measuring them.

Again, I was very impressed several years ago now when I went to the Cavendish laboratory in the cellar of the Cavendish laboratory, sort of Frankenstein like. Apparatus is there, and I was led in to see how they were experimenting on a single atom. And not just on the atom, they were making it do things. They were making it jump through hoops. They were making it do quantum computations on a single qubit. And I looked at the wall, and the wall was covered with graphs of the errors. So, you could regard the whole experiment as an experiment about the errors that happen when you try to make a qubit. And if they hadn’t had those, if they just set up the experiment as it’s later going to be described in the paper, without making the improvements to remove those errors, they could have got random results, they would have probably got the results they were hoping for. That’s what usually happens when you do a bad experiment.

Bad philosophy in quantum theory

Naval: This showed up again recently in the whole room-temperature semiconductor hunt. Where if you want to believe something, and then you drop all of the skepticism that you should have around the measurement, then you can get almost any result you want in a non-replicable way. Or you can see almost anything you want to see. Speaking of which, some things are obvious yet unseen. I think we’re just getting to the fourth strand now, quantum theory, which I think is the one that actually people understand the least. It’s just considered the most esoteric of the disciplines. Normally, most people wouldn’t even dare say they understand quantum theory because of the rigor in the physics that they think is required. But where do you think people get this one? Maybe they approach it from the wrong direction or they’re missing something in plain sight.

David: Now that I think of it, the misconceptions about quantum theory, although in some ways they resemble misconceptions about other theories that are far from everyday experience, like relativity, cosmology, black holes, and so on. In some ways, it’s just unfamiliar, and therefore, people hear what they expect to hear and then double down on their misconceptions. So that happens in all the fundamental theories. But the basic thing that’s gone wrong with quantum theory, unlike the other three strands that we’ve discussed, began inside physics itself. And it is the doubling down by physicists on their misconceptions, which has then been transferred to the public.

And I think we might even have got to the stage now, when the public, I don’t know, maybe I’m being unfair, but I was going to say that maybe the public by now have got a better handle on what kind of a theory many universes quantum theory is than the physicists who still resist it, who are the majority, because the physicists who have resisted have been led by their education, by very strong peer pressure and authoritative pressure and mistreatment of students and their questions and all sorts of nasty things have come together there to make people use bad philosophy as a defense of their misunderstanding of the science of quantum theory.

So instrumentalism and positivism have their stronghold now in theoretical physics. There are very few, if any, philosophers who still defend those things. Even behaviorism. There’s all sorts of moves you can make along the road to avoid the conclusion that reality consists of many universes plus other things, that physicists are more driven to take those than ordinary cranks.

So what is the mistake? As I say, I think originally there was only a small community of physicists who originated quantum theory and there was a little subculture there and that subculture happened to be susceptible to a form of positivism that was worse than the ordinary positivism in that it was also susceptible to kind of mysticism. So all this stuff about, for example, the observer’s consciousness changing the nature of reality, that was not originally in the bad interpretations of quantum theory, Bohr’s interpretation. Bohr never said that. And Niels Bohr, he said a lot of things that were bad philosophy, but he didn’t say that.

And so what has been built on that foundation, an attempted foundation to secure the single universe worldview, has incorporated positivism and then instrumentalism and mysticism and a bad form of empiricism, which is “Shut up and calculate,” but there’s also sheer intimidation. I mean, physicists who are working on branches of physics that don’t directly involve taking a position on this are reluctant to take a position on it because it will reduce their standing with their colleagues, journalists, and so on, or at least I think it will. Well, maybe I’m wrong to psychologize. I mean, I don’t really know why these things have happened. 

Brett: It’s a spectacular, though true, conclusion to reach that, you know, you look at any interference experiment, the double slit experiment, and you can conclude on that basis there are many universes. But you’ve also said the existence of many universes is in fact one of the least surprising and confounding things about quantum theory. What are some of the other more counterintuitive parts of quantum theory?

David: Yeah, I think entanglement is much more counterintuitive. By the way, I’ve also said—I thought you were going to ask how counterintuitive is quantum theory compared with, say, relativity. I think relativity is much more counterintuitive than parallel universes, because parallel universes—they make movies with parallel universes in the plot. It’s very hard to make movies with curved spacetime in the plot.

Naval: I think Interstellar is the only one—

David: Well, yeah. But even that, they avoid the curved spacetime bit. They have the black holes.

Naval: Yes, and they do the accelerated time bits.

David: Yeah. So, that’s a theory that is hard to get your head around, and is very, very different from our experience. And even where now that we have the GPS system over our heads, measuring our positions, many times more accurately than you could if you didn’t take relativity into account, people want to adopt relativity only instrumentally. But they don’t go into flights of fancy like is done in quantum theory. That is the thing that seems to only happen in physics, in quantum theory, and I can’t explain it.

Naval: It seems that the appeal of anthrocentrism, where we’re at the center of everything, is so strong that it sort of re-emerges now under the guise of the observer.

David: Maybe it’s that, but from your own example, people did accept that from modern astronomy. So, in the 20th century, we discovered that even the galaxy is just one among many galaxies, and people were shocked. But they, thinking people, reacted by thinking, “Okay, well, I was wrong, not only are we not the center of the solar system, but we’re not the center of the universe, and we’re not the center of anything.” And now they’re shocked by saying, “Well, in a sense, we are the center of everything”.

Free will is intimately connected with knowledge-creation

Naval: So let’s get into that. So these four strands of the fabric of reality—four theories—they form, for lack of a term, the theory of everything. What now emergent principles and concepts can we talk about that rely upon two or more of them? We were talking about one earlier. Let’s just get a little more formal about it: Knowledge.

David: Yeah, by the way, it’s the theory of everything known. So there are things, glaring omissions in what we know.

Naval: For example, we don’t understand consciousness, we don’t understand creativity, we understand maybe how knowledge grows, but we don’t understand where it comes from.

David: Yes, exactly. So we don’t understand those things, and they’re not part of the four strands, which is The Fabric of Reality, the book, I mean, they are part of reality. So, your question was? Was it connections between the four?

Naval: Yeah, connections between them, and so that can be knowledge, that can be wealth, that can be optimism, that can be error-correction, but there are all these principles. I even suspect the fun criterion, Taking Children Seriously, universal explainer arise out of these.

David: Wow, that’s a lot of things. Let me point out that of the four strands, I invented none. And of the connections, of the two way connections between the four things, I invented one.

Naval: Quantum computation.

David: Yeah. So, The Fabric of Reality is really a sort of, what do you call it, a riff on these things, on these ideas which are true, but haven’t been appreciated and whose connections haven’t been appreciated. So with free will in the multiverse, I think what I said about that in The Fabric of Reality is very inadequate and possibly misleading. I did not mean to say that the multiverse solves any problem of free will. I just kind of used the multiverse as an example to show that Newtonian mechanics doesn’t violate free will either. Those are separate issues and that you can make sense of counterfactuals, whether the world is deterministic or not. By the way, counterfactuals and Constructor Theory is yet another spin off of these things.

Naval: Let me step back. You understand these things at a core level. They inform how you operate in your own life. So you don’t have to get specific about your own life, but [what are some] principles that you sort of know to be relatively true or they are our best knowledge today because of these four strands.

David: So one of the sort of spin offs in regard to free will is that although we don’t know how knowledge-creation happens, free will seems to be intimately connected with knowledge creation so that there’s a lot we don’t know. But again, the argument that because of physics, let’s say, free will can’t possibly exist is just wrong. It’s just misconstruing physics. And one of the things it misconstrues is that, again, because of empiricism and that kind of error, it is thought that all explanations have to fundamentally boil down to predicting things from first principles. So if you can’t predict a thing from first principles, then your theory of it can’t be fundamentally true. And it might be an illusion, and that’s what people think theories of free will amount to. That free will is just an illusion that we create, tell ourselves, but doesn’t correspond to anything at the lowest level. Well, the second law of thermodynamics doesn’t correspond to anything at the lowest level either.

You can’t look at an atom moving in the air, or a molecule moving in the air, and say that molecule is moving irreversibly. None of them are. They’re all moving reversibly. And yet, the combination of them is moving irreversibly. And there’s a theory of that, a hard scientific theory of it, which, if you try to violate, you are a crank. So that’s an example of the fact that scientific knowledge exists on several levels of emergence, although emergence is actually only one of the ways in which high level theories can be related to low level theories. But, in general terms, you can call it emergence. Once you make this mistake, and you say there’s no free will, that can have drastic implications for other high level theories, such as theories of morality.

So some people say, “Well, we’re all made of atoms, and we can’t help what those atoms do. Therefore, murderers are no different from other people. So we shouldn’t be putting murderers on trial or sending them to jail,” or whatever. And on the other hand, the inverse of that argument is, “Well, because a murderer has murdered people, there must be something in their atoms, in their arrangement of their atoms, which makes them a murderer. And therefore, they are to be kept in jail forever, basically, because we can override some things in our genome, but most of the time we are slaves of our genome”. Because of empiricism, because of no free will, and because of all these mistaken and/or bad philosophical theories, we end up ending up with policy backed up by rubbish arguments.

Now, there may be, there are excellent arguments for letting people out of jail, for putting them in jail, for doing x, y, or z to them while they’re in jail, or when they’re not in jail, and all those arguments are perfectly valid domain of philosophy, and some people pursue that kind of philosophy. And one could be wrong, one could be right, one could be half right and so on, but to pontificate about it from the perspective of basically physics is, what shall I call it, is category error. It’s just wrong.

Brett: Some say that it’s compassionate to not subscribe free will to people because of exactly what you’ve said: the murderer is a victim and they cannot help but do what they do. There are those who don’t necessarily argue from physics, but from some sort of folk psychology. Maybe not folk psychology, but a certain psychological theory.

Naval: Yeah, I think there’s, if I understand you correctly, and correct me if I’m wrong, please, I want to understand this. I think there’s two things you’re saying here. One is that some theories only emerge at certain levels. They’re not visible or available to you at lower levels. Thermodynamics is an example. Watching a single molecule or atom in isolation will not tell you anything about irreversibility or statistical irreversibility. And that can only be seen at a macro level, so at a higher level. And so, some theories are equally valid and they’re not capable of being reduced any further, but they’re equally valid at their own levels.

David: Even if they can be reduced, thermodynamics is sort of a case of this, or maybe chemistry is a better case. Even when they can be reduced to a lower level, there may be explanations and laws that only exist at the higher level. So we think that chemistry is entirely due to physics. And we can make predictions about chemical properties using only physics. Basically, using physics plus computers to solve the equations. In addition, there are such things as acids, which you can have theories about, and which you can explain the world in terms of where you could not explain the world in terms of the underlying physical reason.

Naval: No matter how much computation power you had?

David: Well, depends what you mean by “No matter how much,”

Naval: Within the universe, within the limits of the universe.

David: Within the universe, is hopeless. There may be a mathematical computation that is enormously bigger than the universe.

Naval: I think what you’re saying is, so people point to, “Well, it’s all particle collisions, right? Particle collisions explain everything. And so because of particle collisions, this man went and murdered another man.” But you could say, “No, there are some things that have to operate at that level of explanation. Because you can’t compute it and also because you can’t understand it.

David: Yes, it’s the latter. It’s the latter that I’m talking about. Because even if you could predict, which you can’t, you still lack the explanation. As I say in The Beginning of Infinity with the domino theory, you could follow through every single domino striking every other domino and work out that this one domino will never fall over and then you will have predicted it. But you still won’t understand anything about prime numbers. You won’t know that it’s due to prime numbers. There are arrangements of dominoes where nobody knows. Nobody will know for the next 10,000 years why a particular domino stands out and not the others. And some cases we will never know. Those cases all involve continually adding dominoes, but that doesn’t change the fact that there is an explanation of things that these dominoes do that doesn’t have to do with dominoes.

Naval: So do you care to give a summary of your best explanation of free will or should we just skip that?

David: It’s a loaded topic. Well, I don’t really have an explanation. I think that free will is intimately connected with the creation of new explanations. Because I think philosophically, the thing we want from a theory of free will, and which doesn’t seem to be present in like Newtonian physics or anything, is the idea of creation, of something new being created. Before we had Newtonian physics, and as long as we still had philosophy, people would talk about the universe having been created by God out of nothing. In some religions or some theories of creation, God creates the universe out of nothing. It’s not that he makes something into the universe, like mud or whatever, some religions say that, but some of them say, “God created the universe out of nothing.” Now, common sense folk psychology says, humans create something out of nothing when they have a new explanatory idea.

And that’s why we make a difference between “person X pushes person Y onto the railway line when they’re both standing on the platform.” Was person X intending to push person Y onto the platform, or was person X himself pushed by person Z? In both cases, person X pushed person Y because of the laws of physics. But, intuitively, we know that the two situations are chalk and cheese. And you might not be able to tell very easily which it was because the eyewitnesses won’t know. So you need other explanatory knowledge, not just observations to tell you which it was. But which it was is considered a real thing in a court of law. And I think that it is a real thing, and the pivot on which this turns is the fact that creating new explanations is creating a real thing. When Einstein solved the problem of how special relativity is consistent with gravity and invented general relativity and wrote down the theory of general relativity, it’s not the case that the theory of general relativity had already been implicit. In Einstein’s brain, or in the world on planet Earth a hundred years before, or in the Big Bang. It had never been implicit anywhere until Einstein created that knowledge out of nothing.

That’s the quintessential act of free will. It’s an act that was created by Einstein and not someone else and not the blind forces of nature either. It was created by him. So people sometimes use a different example, which I think is not a helpful example—people say, “Think of a random number between one and a hundred,” and you try and think of it and then you plot the numbers that people choose, and they’re nowhere near random. If you ask a random number, no one ever says one or a hundred, and so on. And now, trying to simulate a random number generator is the opposite of free will. That’s using an example which is the opposite of free will to illustrate what people mean by free will. What Einstein wrote on that day was unpredictable because no one else had his problem situation.

And without his problem situation, saying it would have taken the age of the universe is a gross understatement. I mean, there’s no way that somebody without that problem situation could have come up with that solution. So that is the quintessential act of free will, the one that emerges out of you, but not predictably. The reason it’s not predictable is not that it’s random, it’s the opposite, it’s the opposite of being random. It’s because it is the solution.

Naval: So there had to be a problem. The problem required a solution. The solution was arrived at creatively. The solution creates knowledge, which is a real thing, which is causal in the environment, and causes itself to get replicated in everything from GPS satellites to rockets.

and continues on and fundamentally changes the nature of the universe that we operate in.

David: Perfect. I couldn’t have said it that well or at least not that fast.

Naval: And this combines all of the strands of The Fabric of Reality that you’re talking about. Because we have a universal explainer creatively creating knowledge and then causing that knowledge to be replicated into the multiverse and in fact the closer Einstein is to being correct, the more the theory of relativity is replicated across the multiverse and it forms almost a crystal structure of knowledge across the multiverse.

David: Yes. Because the other Einsteins in the other universes would have come up, the other universes would have come up with the same theory. Well, very nearly the same theory. And even in the universes where Einstein didn’t exist, somebody would eventually come across that problem and solve it. And then when they did, that would have merged with the other.

Naval: They would have only solved it and only created the knowledge if they had the problem. If they didn’t have the problem, if it was a computer being told what problem to solve and it was a different problem, it would not have solved it. And actually, before Einstein came along, did people even think they had a problem? Did other people think there was a problem?

David: Einstein wasn’t the only one wondering about this problem. At a crude level, other people did have this problem. In fact, as is often pointed out, the mathematician David Hilbert actually went as far as to write down Einstein’s equations after listening to a lecture of Einstein’s. So Einstein told him the problem. He, being possibly the greatest mathematician in the world at the time, went home, took out a bit of paper, jotted down Einstein’s equations, which it took Einstein like years to work out, but he didn’t know what he was writing. He didn’t understand what he had just written down. Other people in the 19th century had thought about the possibility of curved space for a different reason, but Gauss apparently actually went out with lanterns on hilltops and tried to measure whether the angles of a triangle add up to 180 degrees. He couldn’t do it because the accuracy required is one part in 10 to the eight. But one part in 10 to the eight is not that far away from being possible. And I’ve often wondered whether somebody might actually be able to do it now with laser and so on.

Wealth is not a number, it is a set of transformations

Naval: So let’s talk a little bit about knowledge. What is knowledge in your worldview formed by these four strands?

David: In the course of my philosophical meanderings, I’ve settled on several different conceptions of knowledge, which I think all refer to the same thing. It’s just a different way of characterizing what that thing is, but I think they all come to the same thing. What I’ve recently found most helpful, thanks to Constructor Theory, is that knowledge is a form of information which is necessary for a physical transformation. So if a physical transformation will only happen when a certain type of information is there, then I call that information “knowledge” and that nicely focuses on the knowledge in genes and the knowledge in ideas. And there’s other knowledge which is stored knowledge, like in computers or books. Knowledge can be created, but so far the only things we know of that can create it are evolution and human thought. It’s very tantalizing that there were once several species on Earth that could do this. And they all went extinct, all but us.

Brett: Then interestingly, you take that notion that you’ve just mentioned about knowledge under Constructor Theory as being about transformations, but that’s also the word that you use when you talk about wealth. It’s also about transformation.

David: The wealth of an entity, let’s say of a person or of a country or whatever or of the world can be defined as, in constructor theoretic terms, as the set of all the possible transformations that it could bring about (if it wanted to). It’s never going to bring about all those because they’re exponentially more, or exponentially too many of them. It has to have the right problems.

Naval: And if it has the right problems, then it can use the knowledge plus the physical assets that it has to cause physical transformations. And then if it had the right problems and had the right solutions, then it grows wealth. And if in the process it has to make more creative leaps to do so, it grows knowledge, which also grows wealth.

David: Yes. Well, that is true. One thing that this stresses is that wealth can’t be quantified as a number. Wealth is a set. A set of transformations. And so, I can’t say whether, in these sort of absolute terms, whether Mozart was richer in this fundamental sense than Nathan Rothschild. So, Nathan Rothschild had knowledge of banking, which he had created out of nothing, and Mozart had knowledge of musical beauty, which he had created out of nothing.

In both cases, they were improving on previous ideas, like all knowledge does, but they had created something that was not there before, in both cases. But you can’t say that one of them had more knowledge than the other, because the sets overlap. Or don’t overlap.

Brett: You said there, and in fact, that’s the third time. Einstein created general relativity out of nothing. Mozart created his work out of nothing, but I can hear the objections, “Hold on, they didn’t create it out of nothing. There was pre-existing knowledge there that they have used and mashed up together, rather like ChatGPT. So general relativity was just Einstein riffing off geometry that was there but hadn’t been repurposed for physics.” So how do you answer those opponents of this idea of creation?

David: It’s the same, you know, saying that they were only doing that. They were doing that, they were putting together bits of other knowledge and varying them. That’s not all they were doing, because if you try and do that now, you won’t do it. It’s the same argument as saying humans are only atoms. Well, yeah, humans are only atoms and trees are only atoms and so on. And what’s important about humans is not that they’re atoms. One day maybe we’ll download our minds into silicon and not carbon. And then maybe people will be saying, “Oh, we’re only silicon.”

Brett: Civilization just looking around cries out for an explanation. If you’re going to deny that humans are special, then explain why it is humans and not the bees that are creating the—

The principle of optimism

Naval: Or another way to put it, the combinatorics of discovering relativity just by having some tools that’s so large, it’s instead of monkeys with typewriters, now it’s monkeys with calculators. And they’re going to come up with relativity. It’s still an impossibility.

We’ve talked wealth, we’ve talked knowledge. Oh, let’s talk optimism. The Beginning of Infinity starts out with the principle of optimism, or enters into it very early on. This seems to be the fundamental binding principle of that book—the beginning of infinity of the growth of knowledge.

And so the principle of optimism, error-correction, universe explainers, all of these seem to go together. Is the principle of optimism the most important takeaway from all of this? Is it the most important synthesis? Is it the philosophical basis for how we should probably structure our societies and live our lives? And where does it emerge from?

David: Yeah, maybe not my place to say. It is one of the things that people have told me that it’s that concept that makes them understand the book. I didn’t write it with that in mind. For me, it’s more about epistemology and the role of creation in the physical world and so on.

And the principle of optimism was a corollary of that. But all these things are so connected. That you can start almost anywhere and get to the whole thing by seeing that the thing you were originally interested in has got a wider context. This wider context is important. So another way of looking at the principle of optimism is that it’s just the basic principle of Constructor Theory.

There ain’t no one here but us people. If you think of the set of physical transformations that can be brought about and can’t be brought about, of the ones that can be brought about, the overwhelming majority, and again, that’s an understatement, can only be brought about by people. By people who create knowledge because they want to bring that thing about.

And if there weren’t people in the universe, this set of things which can be brought about would be tiny, and again, tiny is an understatement. It would be almost nothing. There’d be like a few dozen things in the world, different kinds of stars, and that would be that. So that’s an entry point to the idea of optimism, because you can see that if something is possible, then either it’s going to happen spontaneously, like a star or a black hole, or it’s going to be brought about. And if it’s going to be brought about, it’s almost certain to have been brought about by knowledge. And knowledge will have been created either by evolution or almost certainly in the long run, by people. Because at the moment we’re kind of maybe still catching up or have just moved ahead of evolution in the sophistication of the knowledge that we have.

Naval: So I think what you’re saying is that if you look at the inanimate universe, there’s only a few fundamental forces acting in a few known ways. And although it’s replicated at huge scales, the diversity of knowledge there is quite low. And then after that comes the knowledge creation through evolution, which has had a long time to work relative to humans.

So there are some impressive things to show like grass and trees and humans themselves. But the growth of that rate of knowledge is very low. And you can imagine human knowledge growing much faster. And once it begins to spread amongst the stars being the primary thing that needs to be explained to explain the structure of the universe. And so, we are at the beginning of the growth of an infinity of knowledge because we have humans now creating knowledge at a rate and diversity that’s much greater than anything that’s come before. And because we’re universal explainers, anything that can be explained, we can explain. Anything that can be created, we can create. Well, I’m optimistic.

David: Any transformation that is physically possible can be brought about and it requires knowledge.

Constructor Theory

Naval: So what is the problem you’re trying to solve with Constructor Theory?

David: We get into physics here. So there are several areas of physics. There is a sort of tacit consensus among people who study fundamental physics in the sense that they’re looking to find out what the laws of nature are, that level of fundamental, that a theory of physics or in general, a theory of science in general, it consists of a theory that says there are initial conditions, there are laws of motion, and to understand the universe you have to understand what the initial conditions are and what the laws of motion are. And everything else is derivative. If such people believe in explanation, they can make the same story about explanation as well. Maybe a little less plausibly. But, the trouble is that not all scientific theories are like that. In fact, in some sense, most scientific theories are very unlike that. My favorite example is Darwin’s theory of evolution.

It’s not the theory that predicts the existence of elephants. It’s a theory that explains the existence of elephants. And the explanation does more than any prediction possibly could. Like, if you could somehow run a supercomputer about the plains of Africa where elephants evolved and it predicts, you know, at one point it prints out a picture of an elephant. It hasn’t explained anything about what has happened or why. Whereas Darwin, who did not have a computer, could have had one. He could have had one if Babbage had pulled his finger out. Darwin understood it, despite not having a computer. So, Constructor Theory tries to get out of this class distinction among the sciences, among knowledge, and so on, and tries to make a uniform framework in which laws of physics and scientific laws in general, and even beyond science, can be expressed.

By the way, another thing that really stands out like a sore thumb in the prevailing way of looking at fundamental science is that it’s not symmetrical in time. So it says that there are initial conditions and laws of motion and everything is then evolved forward from the initial conditions. But the laws of motion are time symmetric, so you could just as well start at the end of the universe and say, we need a theory of the end of the universe. Well, the end of the universe, we hope, is going to contain lots of knowledge, perhaps an infinite amount. And there’s no way we can form a theory of that by definition. I mean, that’s the thing we can least form a theory about in the whole of creation. And even worse, we could also frame the prevailing theory as to understand the world, you have to understand it today on a certain day in October, and then we can use the laws of motion to work backwards and forwards to understand the rest of the universe.

Well, this is no good. I mean, this obviously can’t explain anything. And it’s really exceptional that initial conditions and laws of motion are useful at all in explaining things. So it happens it was useful in explaining the solar system and so on, and it’s useful in making microchips and understanding how to make drugs and chemistry and so on. So those are actually big fields, but compared with what we have yet to know, that isn’t much, and some of the things we already know are already not in that form. Like thermodynamics, for example.

Naval: Is it fair to say or too much of a leap that a good explanation should be timeless? Or relatively timeless. It shouldn’t be so dependent upon the “t” variable.

David: It shouldn’t depend on a specific time, yes, absolutely.

Naval: And so to the extent that you’re talking about initial conditions and laws of motions, it’s very time bound. It seems very restricted. And you can’t go too far forward because you don’t know the growth of knowledge coming up by definition. And going backwards is too reductive. It reduces down to, “We only know it at that point in time,” we don’t have the underlying explanation. It’s as you said, going back and saying we understand the universe a billion years ago is like spitting out a picture of an elephant. The explanation should apply to any species at any time. And so Constructor Theory is, in that sense, it’s trying to be more timeless and less predictive and more explanatory.

David: Yeah, it has no opinion about predictive, but it wants to be explanatory. Sometimes predictions are part of explanations, so yeah, it wants to explain. And the idea in Constructor Theory is not to look at a physical system in isolation as if nothing else existed and say, what will it do? We have these equations that say what it will do.