"Text generation" undersells it a little bit. What are humans except "text generation" machines? Language is the stuff of reason. GPT-3 has demonstrated capabilities that we believed were exclusive to humanity --- humor, logic, sarcasm, cultural references --- in an automatic and generalizable way. It's far more than a "text generation" system. I look forward to seeing what GPT-4 and GPT-5 can do. I suspect we're all going to be amazed by what we get when we continue down this path of simple scaling (and sparse-ification) of transformer architectures trained on (basically) the whole internet.
> What are humans except "text generation" machines?
The ability to grow and choose our own direction: to choose what our goals are, curiosity, self-awareness, desire. To imply that GPT-3 is anything close to strong AI is kind of ridiculous.
No, you can teach it a new task or to do multiple tasks, but it will never be able to independently identify what a new task might be and learn to do it. An important distinction when talking about AGI.
And how do we indicate that any of these processes has occurred except through the medium of language? A sufficiently good text predictor is a sentient mind. I don't believe that there's any experience of consciousness distinct from the use of language.
I disagree with this... I have a hunch GPT-3 is still falling short PRECISELY because of its dependence on language, and that it actually is going to great lengths to overcome this design flaw to create the simple texts that we're all fawning over.
I predict within a few years, the descendants of GPT-3 will use very different fundamental units for processing that differ greatly from the current state-of-the-art (i.e. they won't use BPEs and their ilk anymore, except for final output) and will be far more powerful as a result.
The descendants of GPT-3 will be using the comments on this page to get new ideas.
I do agree with you. We, as somewhat intelligent beings, do not base our thinking on words or language AFAIK, even though it's our best way to convey ideas to others. And we learn through experience, way faster than GPT-3 does, with fewer shots. It looks like the attention mechanisms are what made these models actually start to understand things... But those attention mechanisms are still very raw and mainly designed to be easy to execute on current hardware, I wonder how fast will we refine that.
Finally it looks like, once trained, these models don't learn when we use them. It definitely doesn't learn through experience and that's a major limitation on how intelligent it can be.
For most of human history text has not existed. I think you are conflating language with written language which is quite common in the post-Gutenberg age.
I think sentience like most things is a spectrum, so I'm not really sure what you mean by sentient, but I would argue that for most people the bar for sentience is much higher than text prediction. The Chinese room is only one aspect of our minds, and we don't even know what consciousness is.
I have to disagree with this perspective and reiterate my original position: language is sentience.
And to be fair, reasonable people stake out positions on both sides of this debate: I'm not claiming that the alternative proposition is somehow unreasonable. It's a legitimate subject of scholarly disagreement.
Nevertheless, I'm still firm on language. Why? Because all complexity is ultimately about symbolic manipulation of terms representing the process of manipulation itself. ("Godel, Escher, Bach" is a fantastic exploration of this concept.) How can you manipulate concepts without assigning terms to their parts? That's what language is.
The question I like to ask is this: are there any ideas that you cannot express using language? No? Then how is thought distinct from language?
Yes, people (myself included) experience a "tip of the tongue" experience where you feel like you have an idea you can't just yet express. But maybe this experience is what reason feels like. Why should idea formation take only one "clock cycle" in the brain? Why should we be unaware of the process?
I think this feeling of having an idea yet being unable to formulate it is just the neural equivalent of a CPU pipeline stall. It's not evidence that we can have ideas without language: it's evidence that ideas sometime take a little while to gel.
I think you only need language to communicate, it’s not necessary in order to think. Do you agree that a human that grows up in isolation probably won’t develop a language? Would you say such a human isn’t sentient?
I think as highly social beings we often annotate all of our thoughts with the language we could use to communicate them, which could lead us to believe that the thoughts are indistinguishable from the language, but that conclusion seems like an error to me. I’ve also heard some people talk about how they are “visual” or “geometric” thinkers and sometimes think in terms of images and structures without words.
Assuming Genie the feral child was not born mentally retarded, it may suggest that language is critical for human level intelligence. There's also the theory in anthropolgy which I believe has some evidence that human intelligence exploded with development of more complex language.
I think you're mixing up sentience/consciousness/intelligence. Many animals are sentient for example but as far as we know, they don't really have language. But I think I get what you're getting at, which I believe is "human level intelligence requires langauge". I think that's a reasonable take. But you said "sentience", and you said "is", which makes your position difficult to agree with.
Hold up, you can't just throw out a claim like "many animals are sentient" as if it's a statement of fact. You might be right, but there's a reason that "the hard problem of consciousness" is hard. We don't really have any way to distinguish sentience/non-sentience based on behavior. The whole concept is extremely mushy.
You're right but as I've stated before, sentience and consciousness are different terms, and sentience has a definition in which the idea that animals are sentient isn't all that controversial. Not a mathematical axiom sure, but it all depends on what you mean by sentience, and I'm going by the classic definition.
Yes. You're right. I was sloppy with language. To be specific, I think that "human level intelligence" is basically synonymous with "able to think about thinking", and I think to do that, you need symbolic manipulation, and language is the only way we can do symbolic manipulation.
I don't believe we have any evidence this edition of GPT is capable of reasoning. I haven't experimented with it but I doubt it will respond correctly to even simple logic puzzles provided they are framed in a novel way (it may have already seen some puzzles, but I doubt it can extend that knowledge to a puzzle with different words)
> I don't believe that there's any experience of consciousness distinct from the use of language.
Not sure there's one I can communicate to you, but I'm perfectly capable of forgetting the word for something and still knowing unambiguously yet wordlessly what it is, that's an experience.
Catching a ball? Running? Experiencing emotions from wordless music? Viewing scenery? Engaging with a computer game? How are they not conscious experiences?
> I don't believe that there's any experience of consciousness distinct from the use of language.
To me this indicates a very narrow view of consciousness. Consider for a moment the quiet consciousness of the cerebellum for example.
I like the way David F. Wallace put it: 'Both flesh and not'. There's an astounding amount of consciousness that is not bound by language. One can even argue that language might hinder those forms of consciousness from even arising.
If you believe in consciousness divorced from flesh, you're in strong intellectual company, but that's not a path I can go down. Metaphysically, I just can't accept the idea that the mind is anything but the computation performed by the brain.
I rather agree with you, but the interaction of the mind with the physical reality is also extremely important to shape it. GPT3 has no interaction with a physical world. Any formal system that cannot interact with something outside of itself will be intrinsically limited, for one thing by Gödel incompleteness theorem.
I agree about the need for an environment. The difference between GPT-3 and an agent in an environment is that GPT-3 only saw tons of static text, while an agent can design an action (an experiment), act it out and observe the results, drawing conclusions. Thus it can act in a way similar to the scientific method.
In the book the meaning of the quote is more to the effect of how truly great athletes can perform on the verge of what spectators would consider inhuman or possible. I'd be hard-pressed to believe that language and syntax would be responsible for these kinds of actions and flow. I'd argue that getting into such a state is not possible while the mind is caught up in the language of things rather than the experience itself, and reacting to it directly. This is what I meant by the quiet consciousness, devoid of language or syntax.
I don't think any modern cognitive scientist believes that the statement "language is the stuff of reason," even allowing for poetic flair, is meaningfully true (and I'll leave aside that humans are, obviously, much more than text generation machines.) GPT-3 can generate text but the only context it has is its prompt; when humans generate text they have their complex situation in the world (and perhaps even non-worldly factors, e.g. apprehension of mathematical reality) as context. Fitting the latter kinds of context into AI models is the challenge still facing the path to AGI.
I agree with you, but also the following is "just" an implementation detail:
> only context it has is its prompt
The only real context is its latent representation of the prompt, there's nothing fundamentally limiting visual, auditory, symbolic, and mixed prompts as long as they map to a common latent space and the generator is trained on it.
>What are humans except "text generation" machines?
Text generation doesn't chop wood, optimize speedruns, build machinery or win 100-metre dashes.
Text may be involved in training for these things, but to say that doing them is text generation would be like saying that... since compiling code and running AlphaZero both generates bits, AlphaZero is a compiler.
The ability to do these tasks is neither necessary nor sufficient for recognizing something as human. Helen Keller was human after all. What differentiates us is language.
"Text generation" undersells it a little bit. What are humans except "text generation" machines? Language is the stuff of reason. GPT-3 has demonstrated capabilities that we believed were exclusive to humanity --- humor, logic, sarcasm, cultural references --- in an automatic and generalizable way. It's far more than a "text generation" system. I look forward to seeing what GPT-4 and GPT-5 can do. I suspect we're all going to be amazed by what we get when we continue down this path of simple scaling (and sparse-ification) of transformer architectures trained on (basically) the whole internet.