Why AI Feels Intelligent (and Why That’s Misleading)

The short version

AI feels intelligent because fluent language makes learned patterns look like human understanding. The comparison breaks at the learning process: LLMs optimize next-token prediction over text, then receive limited steering, while humans learn continuously through bodies, goals, social feedback, and lived experience. Similar outputs do not establish shared meaning, intention, consciousness, or a human relationship to the world.

• LLMs and humans both learn patterns, but the signals, grounding, and relationship to the world are very different.
• A model can imitate language impressively without having human experience, goals, or understanding behind the words.
• The confusion comes from judging the output surface instead of the learning process that produced it.

Watch the full video here: 

In this article, I want to come back to a comment I got under a short where I said that LLMs learn by copying patterns, and humans also learn by copying people, but it’s not the same. The comment was basically: how is it not the same?

That’s a fair question, because if you say it without much more detail, as I did, which isn’t really ideal, it does sound like LLMs and humans do the same thing. Humans imitate. Models imitate. That’s it.

But the reason this comparison keeps causing confusion is that it mixes two very different levels of description. From far away, we know that both humans and LLMs are pattern learners. But when we go closer, the learning process, the signals, the constraints, and the connection to the world are completely different. And we understand why it’s so far from consciousness or what we call AGI. If you don’t separate those layers, you end up with statements that feel right but don’t actually explain what’s going on, and you may think that AGI is coming for your job within a few months.

Spoiler: for the vast majority of us: it isn’t.

You’re safe. At least for 2026.

So I want to walk through this carefully, end to end, with one single example the whole time, but I have to split this explanation into three parts that I’ll define when the time is due. First, how LLMs actually learn during pretraining. Second, what fine-tuning and reinforcement learning change and what they don’t. And third, how humans learn language and stories, and where the resemblance really stops.

Let’s start with pretraining, because this is where most of the confusion comes from, and it’s the most important part of training a powerful language model.

During pretraining, a large language model is trained with one core objective: given a sequence of tokens, predict the next token. That’s it. You hide a token, the model guesses it, you adjust the parameters, and you repeat this over enormous amounts of text. There’s no notion of meaning, intention, or communication in the objective itself. The model is not trying to tell a story. It’s not trying to be helpful. It’s not trying to understand anything. It’s minimizing prediction error.

And I want to emphasize that I’m really talking about tokens here, not words. Models don’t know words; they know numbers. We represent every word of the language in their respective index, which we call a token. It’s obviously a simplified way to see this, in reality, we learn to optimize these tokens based on how often the words appear in the English language and all, but it all comes down to this same idea that the language, from a list of numbers, learns to predict the next number.

So if the model sees “The kid looked under the bed and”, which actually looks like this [14305, 10585, 7111, 1234, 279, 4950, 323, 863], it assigns probabilities to what comes next. “Found” [6788] is likely. “Nothing” [24714] is possible. “Volcano” [37461] is almost impossible. Over time, by seeing billions and trillions of these contexts, it learns extremely rich regularities about language. Syntax, style, narrative structure, even a lot of world knowledge, because that knowledge is encoded in text, or rather in these tokens.

Scale is doing a huge amount of work here. Modern models are trained on trillions of tokens. A human, even an extremely well read one, will see orders of magnitude less linguistic input over a lifetime. We’re talking millions, maybe hundreds of millions of words, not trillions or even more. That difference in scale is not a detail. It’s why such a simple objective can lead to such complex behavior even from a non-conscious machine. It’s also the main difference between models before and after ChatGPT.

Now, people often push back here and say: but humans also predict the next word when reading or listening. And that’s true. There’s strong evidence from cognitive science that humans form expectations about upcoming words and that predictable words are processed faster. And I do this a lot, which annoys many of my close ones. So yes, prediction exists in both systems.

This is the first real resemblance. Both humans and LLMs use prediction. Both improve with exposure. Both benefit from patterns.

But here’s the key difference that everything else builds on.

For the model, prediction is the final goal. For humans, prediction is just a side effect of a true understanding.

When you listen to a story, you’re not trying to guess the next word for its own sake. Even worse, fast readers actually skip words. LLMs could never do that. We are actually building a mental model of what’s happening. You’re tracking characters, intentions, emotions, causes and consequences. Creating a new world in your head where things move and evolve. The predictions fall out of that process because language is structured in this mental model. For the LLM, there is no separate layer where understanding lives independently of the prediction task. If something like meaning exists inside the model, it exists only because it helps predict tokens.

This is why experts like Yann LeCun have been so vocal about the limits of purely text based models. His argument is not that they’re useless. It’s that they’re missing key ingredients of how humans and animals learn: grounding in the physical world, multimodal perception, interaction, and the ability to build explicit world models through action and feedback. From that perspective, next token prediction on text is powerful, but it’s not the same learning problem humans are solving.

Other researchers, like Ilya Sutskever, take a more optimistic view. The argument there is that language itself encodes a huge amount of structure about the world, and that to predict language well at scale, a model may be forced to internalize increasingly abstract representations that function like world models in what we call its latent space. It’s still an open debate and we’ll only know with time, but I must say that current LLMs seem very far from developing a true understanding, even though scaling has proven to provide incredible new skills we didn’t necessarily expect before trying to push these trainings to that scale.

But regardless of where you land on that debate, the mechanism today is clear. Pretraining is about compressing patterns in text. Not interacting with a world. Not forming goals. Not testing hypotheses. Just absorbing statistical regularities.

Now let’s anchor this in a simple concrete example so this doesn’t stay abstract.

We can take a super simple story: a kid loses their dog, searches all day, and finds the dog at night.

A pretrained model can generate a very convincing version of this story. It knows that stories like this usually have a setup, tension, and resolution. It knows what kinds of words tend to appear in emotional moments. It knows how paragraphs usually flow.

But it’s not choosing those words because it wants to convey sadness or relief. It’s choosing them because, given the prefix, those words have high probability under its learned distribution. The story structure emerges from local token decisions made at massive scale.

Let’s see with an even more concrete example, instead of sticking in the abstract world of languages.

When a human learns to paint or play the piano, imitation is involved, of course, but it’s not microscopic imitation. You don’t copy every brush stroke exactly. You extract techniques. You form concepts like perspective, balance, contrast, and emotion. You copy the end result. You can even then apply those concepts to scenes you’ve never seen before.

An LLM, during pretraining, is much closer to copying exact strokes than extracting intent. It learns how small local moves tend to follow other local moves. Token after token. Pixel after pixel in image models. The global structure comes from the accumulation of these local rules, knowing what stroke to do next on the painting from all those that it did before, not from an explicit understanding of composition or meaning.

That doesn’t make the model dumb. It simply makes it different.

Now let’s move to the second phase, because this is where a lot of people unintentionally attribute human like qualities that don’t actually come from pretraining.

After pretraining, you don’t yet have a helpful assistant. You have a model that’s good at continuing text. Fine-tuning and reinforcement learning are what shape that raw capability into something that follows instructions, answers questions, and behaves in socially acceptable ways. It basically is the difference between this powerful autocompleter into what you use inside ChatGPT.

In this process, humans are paid to show the model examples of desired behavior. They rank outputs. They reward clarity, helpfulness, and safety. The model is optimized to produce more of what humans prefer.

This changes behavior a lot. It’s why the model sounds polite. It’s why it explains its answers. It’s why it avoids certain topics.

But it’s important to understand what this phase is not doing. It’s not giving the model new grounding in the world. It’s not giving it lived experience. It’s not teaching it what it feels like to lose a dog or to convey a touching story about it. It’s shaping output behavior on top of representations learned during pretraining. This is the part that most resembles human learning on the surface, because we also learn from feedback. Yet it usually requires far more examples than we ever need, and more importantly, the feedback signal is very different. The model is not rewarded for how it reasons or why it made a decision, only for whether the final output matches what humans labeled as good or bad.

If you try to learn a climbing move, you learn through trial and error too. You fall, you adjust your grip, you shift your weight, you feel tension in your body, you build intuition about balance and momentum. That feedback is continuous, embodied, and deeply tied to the process itself. You’re not just told “success” or “failure” at the end and then adjust. You feel what went wrong as it’s happening, and your internal model updates moment by moment.

In contrast, reinforcement learning in LLMs is mostly outcome-based and external. The model doesn’t experience the attempt. It doesn’t feel instability or effort. It doesn’t know which internal steps mattered. It only gets a signal that the final sequence of tokens was preferred or not. So while both systems use reinforcement, humans learn by interacting with the world and updating rich internal models through experience, whereas LLMs are nudged statistically toward outputs humans like, without access to the underlying process that led there.

That’s why this phase can make models behave more human-like without actually making their learning human-like.

Now, this is also the right moment to talk about reasoning, because this is where people often say, “Okay, but the model thinks. You can see it reason step by step.”

What’s actually happening is more subtle.

When an LLM appears to reason, it’s still doing the same core thing: generating the next token based on the previous ones. The difference is that it has seen a huge number of examples of reasoning-like text. Explanations, math steps, arguments, proofs. It has learned that for certain kinds of questions, producing intermediate text that looks like reasoning tends to lead to better final answers.

So when you see the model “thinking,” what you’re seeing is not an internal planning process deciding which steps to take like we do when we think. You’re seeing language that resembles reasoning because that kind of language has high probability in that context.

This is why reasoning can feel impressive and fragile at the same time. The model doesn’t know which steps are necessary. It doesn’t know which ones are sufficient. It’s generating sequences that look like reasoning, and sometimes those sequences line up with correct logic, and sometimes they don’t. And scaling allows us to try many routes in parallel and converge towards the best one and look intelligent.

For humans, this is flipped. You usually reason first, often silently, sometimes visually or abstractly, like creating a world model that you see yourself in, like mentally seeing yourself do a specific climb, and then you use language to express the result of that thinking, or in the case of climbing, you grab the holds and start. Language is mostly a downstream of thought. For LLMs, language is the thought. There is no separate layer where reasoning happens and then gets translated into words. This is just decided by OpenAI on which token to keep as reasoning ones and which to send you to see.

That doesn’t mean models can’t solve real problems. Clearly they can. It means the mechanism is different, and that difference explains both the strengths and the failure modes.

Now let’s come back to humans, because this is where the deepest difference lives.

Humans do imitate. That part is real. We borrow phrases. We copy styles. We learn by exposure.

But human learning is grounded from the start. Words are tied to perception, action, and social interaction. You don’t learn what “dog” means just by reading it. You learn it by seeing dogs, hearing them, interacting with them, being corrected, and embedding that word into a broader model of the world.

Humans also learn with agency. You want things. You try to achieve goals. You fail. You adjust. You ask questions. Learning is driven by curiosity, survival, and social connection, not by passively minimizing a loss function over a static dataset.

I have yet to see an LLM with the curiosity to ask me something unless it was programmed to do so or prompted to do so.

Humans are also incredibly data efficient. A child can learn a new word from a handful of examples. A model might need thousands or millions of occurrences. Scale compensates for this, but the underlying process at their core is still different.

And humans learn continuously. Your model of the world updates every day. Most LLMs are trained, frozen, and then lightly steered. They don’t accumulate new understanding in the same open ended way.

So when someone says, “LLMs learn like humans because both copy patterns,” the precise answer you should give them is this:

“Check out this article by Louis-François and you’ll get it!”

Ok seriously, the real answer should be something like:

Yes, both rely on patterns. Yes, both involve prediction. Yes, imitation exists in both. But humans learn patterns in order to build meaning, act in the world, and pursue goals. LLMs learn patterns in order to predict tokens, through trillions of trials, and only later are shaped to behave in ways humans find useful through millions of trials. That’s why the outputs can look similar while the learning processes are fundamentally different.

I hope the difference is a bit clearer now! And if you’d like, I could go even deeper into the reasoning question in another article, because that opens up a whole discussion around chain of thought, hidden reasoning, tool use, and why certain problems still break models so easily. Let me know if you’d like to see that next in the comments!

Until then, thank you for reading the whole article, and I’ll see you in the next one!

FAQ

Why does AI feel intelligent?

It produces fluent, context-aware language that resembles human explanation, which makes the behavior feel more grounded than it is.

Why is the human comparison misleading?

Humans learn through bodies, goals, social feedback, and lived experience. LLMs learn from text patterns and optimization.

What should builders remember?

Do not infer understanding from fluency. Test the system on the actual task and the failure modes that matter.

Does next-token prediction create meaning?

It creates representations useful for prediction, but that does not establish human experience, intention, or understanding.

Why do LLMs need so many examples?

They learn statistical regularities from data rather than bringing the embodied experience and prior knowledge of a person.

What does fine-tuning change after pretraining?

It shapes how the model responds and follows human preferences, but it does not give the model human experience or a grounded understanding of meaning.