Wondering if Modern LLMs like GPT4, Claude Sonnet and llama 3 are closer to human intelligence or next word predictor. Also not sure if this graph is right way to visualize it.
Shouldn’t those be opposite sides of the same axis, not two different axes? I’m not sure how this graph should work.
Lemmy is full of AI luddites. You’ll not get a decent answer here. As for the other claims. They are not just next token generators anymore than you are when speaking.
There’s literally dozens of these white papers that everyone on here chooses to ignore. Am even better point being none of these people will ever be able to give you an objective measure from which to distinguish themselves from any existing LLM. They’ll never be able to give you points of measure that would separate them from parrots or ants but would exclude humans and not LLMs other than “it’s not human or biological” which is just fearful weak thought.
I’ll preface by saying I think LLMs are useful and in the next couple years there will be some interesting new uses and existing ones getting streamlined…
But they’re just next word predictors. The best you could say about intelligence is that they have an impressive ability to encode knowledge in a pretty efficient way (the storage density, not the execution of the LLM), but there’s no logic or reasoning in their execution or interaction with them. It’s one of the reasons they’re so terrible at math.
Somewhere on the vertical axis. 0 on the horizontal. The AGI angle is just to attract more funding. We are nowhere close to figuring out the first steps towards strong AI. LLMs can do impressive things and have their uses, but they have nothing to do with AGI
There’s a preprint paper out that claims to prove that the technology used in LLMs will never be able to be extended to AGI, due to the exponentially increasing demand for resources they’d require. I don’t know enough formal CS to evaluate their methods, but to the extent I understand their argument, it is compelling.
can you give an example of any third data point such as a rock or a chicken
rockegg
Intelligence is a measure of reasoning ability. LLMs do not reason at all, and therefore cannot be categorized in terms of intelligence at all.
LLMs have been engineered such that they can generally produce content that bears a resemblance to products of reason, but the process by which that’s accomplished is a purely statistical one with zero awareness of the ideas communicated by the words they generate and therefore is not and cannot be reason. Reason is and will remain impossible at least until an AI possesses an understanding of the ideas represented by the words it generates.
That’s literally how llma work, they quite literally are just next word predictors. There is zero intelligence to them.
It’s literally a while token is not “stop”, predict next token.
It’s just that they are pretty good at predicting the next token so it feels like intelligence.
So on your graph, it would be a vertical line at 0.
What is intelligence though? Maybe I’m getting through life just by being pretty good at predicting what to say or do next…
Agreed
This is true if you describe a pure llm, like gpt3
However systems like claude, gpt4o and 1o are far from just a single llm, they are a blend of tailored llms, machine learning some old fashioned code to weave it all together.
Op does ask “modern llm” so technically you are right but i believed they did mean the more advanced “products”
Though i would not be able to actually answer ops questions, ai is hard to directly compare with a human.
In most ways its embarrassingly stupid, in other it has already surpassed us.
That is just next word prediction with extra steps.
None of which are intelligence, and all of which are catered towards predicting the next token.
All the models have a total reliance on data and structure for inference and prediction. They appear intelligent but they are not.
i think the first question to ask of this graph is, if “human intelligence” is 10, what is 9? how you even begin to approach the problem of reducing the concept of intelligence to a one-dimensional line?
the same applies to the y-axis here. how is something “more” or “less” of a word predictor? LLMs are word predictors. that is their entire point. so are markov chains. are LLMs better word predictors than markov chains? yes, undoubtedly. are they more of a word predictor? um…
honestly, i think that even disregarding the models themselves, openAI has done tremendous damage to the entire field of ML research simply due to their weird philosophy. the e/acc stuff makes them look like a cult, but it matches with the normie understanding of what AI is “supposed” to be and so it makes it really hard to talk about the actual capabilities of ML systems. i prefer to use the term “applied statistics” when giving intros to AI now because the mind-well is already well and truly poisoned.
what is 9?
exactly! trying to plot this is in 2D is hella confusing.
plus the y-axis doesn’t really make sense to me. are we only comparing humans and LLMs? where do turtles lie on this scale? what about parrots?
the e/acc stuff makes them look like a cult
unsure what that acronym means. in what sense are they like a cult?
They’re still word predictors. That is literally how the technology works
Yeah, the only question is whether human brains are also just that.
This should just be a 1D spectrum line.
</dataviz>
All A no I
Allanoi is going to be the name of a 0 INT Warforged character I’ll create ^^
Sure, they ‘know’ the context of a conversation but only by which words are most likely to come next in order to complete the conversation. That’s all they’re trained to do. Fancy vocabulary and always choosing the ‘best’ word makes them really good at appearing intelligent. Exactly like a Sales Rep who’s never used a product but knows all the buzzwords.
(-10, 20)
They’re not incompatible, although I think it unlikely AGI will be an LLM. They are all next word predictors, incredibly complex ones, but that doesn’t mean they’re not intelligent. Just as your brain is just a bunch of neurons sending signals to each other, but it’s still (presumably) intelligent.
