Link to original article
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Inference Speed is Not Unbounded, published by OMN on May 8, 2023 on LessWrong.
[The intro of this post has been lightly edited since it was first posted to address some comments. I have also changed the title to better reflect my core argument. My apologies if that is not considered good form.]
This post will be a summary of some of my ideas on what intelligence is, the processes by which it’s created, and a discussion of the implications. Although I prefer to remain pseudonymous, I do have a PhD in Computer Science and I’ve done AI research at both Amazon and Google Brain. I spent some time tweaking the language in order to minimize how technical you need to be to read it.
There is a recurring theme I've seen in discussions about AI where people express incredulity about neural networks as a method for AGI since they require so much "more data" than humans to train. On the other hand, I see some people discussing superintelligences that make impossible inferences given virtually no input data, positing AI that will instantly do inconceivable amounts of processing. Both of these very different arguments are making statements about learning speed, and in my opinion mischaracterize what learning actually looks like.
My basic argument is that the there are probably mathematical limits on how fast it is possible to learn. This means, for instance, that training an intelligent system will always take more data and time than might initially seem necessary. What I’m arguing is that intelligence isn’t magic - the inferences a system makes have to come from somewhere. They have to be built, and they have to be built sequentially. The only way you get to skip steps, and the only reason intelligence exists at all, is that it is possible to reuse knowledge that came from somewhere else.
Three Apples and a Blade of Grass
Because I think it makes a good jumping off point, I’m going to start by framing this around a recent discussion I saw around a years-old quote from Yudowsky about superintelligence:
A Bayesian superintelligence, hooked up to a webcam, would invent General Relativity as a hypothesis . by the time it had seen the third frame of a falling apple. It might guess it from the first frame, if it saw the statics of a bent blade of grass.
The linked post does a good job tearing this down. It correctly points out that there are basically an infinite number of possible universes, and three frames of an apple dropping are not nearly enough to conclude you exist in ours. I might even argue that the author still overstates the degree to which three images could reduce the number of universes in consideration; for instance the changing patterns on a falling apple don’t actually tell you it’s in a 3D world, that would require you to understand how light interacts with objects.
Still, I do think the author successfully explains why, at a literal level, EY’s statement is wrong.
But at a deeper level, this entire framing still feels very off to me, as if even asking that question is making a category error. It feels like everyone is asking what the number three ate for breakfast. It suggests that one could have a system that is simultaneously superintelligent but has absolutely no knowledge about the world at all.
Knowing what we now know about intelligence, I just don’t think that’s possible. And I don’t just mean it’s impractical, or we just aren't capable of building an AI like that. I mean that I believe with very high confidence that such a thing would be a mathematical impossibility.
I think there’s a human tendency to want a certain type of structure to intelligence, and I see this assumed a lot in places like this forum (I was a lurker before I made this account). There’s a desire to see intelligence as synonymous with learning, where existing knowledge is something comple...
view more