Lesson 2 of 8 · 8 min read
In Lesson 1 we established that AI learns patterns from examples rather than following rules someone wrote. That’s the what. This lesson is the how.
Stay with it, because this is the lesson that pays off later. Almost every frustrating thing AI does — inventing a source, contradicting itself, nailing a hard task then failing an easy one — stops being mysterious once you know what’s happening underneath. You’ll start predicting the failures instead of being surprised by them.
No maths. Promise.
Learning by prediction, not instruction
Here’s the whole idea in one sentence: the model is shown a piece of text with the next bit hidden, it guesses what comes next, and it gets adjusted based on how wrong it was.
Repeat that a few trillion times.
Take the sentence “The capital of France is Paris.” The model sees “The capital of France is” and has to guess. Early on, it guesses nonsense. It gets corrected. Slightly adjusted. Next example.
Do this across a substantial fraction of everything humans have written, and something strange happens. To get good at guessing the next word, the model is forced to absorb the patterns behind the words — grammar, facts, tone, reasoning, the structure of an argument, the shape of a joke. Not because anyone taught it those things. Because you can’t reliably predict the end of a sentence without them.
That’s it. Everything ChatGPT does emerges from getting extremely good at one boring task.
What “training” actually involves
Training a large model has three rough phases.
First, it reads. An enormous amount of text — websites, books, code, articles, forums. This is where the raw capability comes from, and it takes months and costs a fortune in computing power.
Second, it’s shown good examples. Raw prediction gives you something that can continue text but not something you’d want to talk to. So humans write examples of good responses — helpful, clear, well-structured — and the model learns to imitate that shape.
Third, it’s given feedback. People compare responses and pick better ones. The model adjusts towards what people preferred. This is what makes it feel conversational rather than like autocomplete.
The important bit: once training finishes, the model is frozen. It doesn’t learn from your conversations. It doesn’t remember you. Every chat starts from the same fixed state.
Why it forgets you
This surprises people, and it’s worth being clear about.
When a chat “remembers” what you said earlier, that’s not learning. The entire conversation is simply being re-sent to the model with each message. It’s reading the whole thread again, every time, and responding to all of it at once.
That window has a limit. Long conversations eventually push the earliest parts out — which is exactly why a model that was following your instructions perfectly starts drifting after an hour. It isn’t losing focus. Your instruction has fallen off the edge.
Practical consequence: for anything important, start a fresh chat and restate the context. Don’t assume it’s holding something from twenty messages ago.
(Some tools now bolt a memory feature on top of this. That’s a separate system saving notes and feeding them back in — not the model learning. The model underneath is still frozen.)
Why it makes things up
Now the payoff.
The model is a prediction engine. Ask it for a source and it produces text that looks like a source — plausible author, plausible title, plausible year, plausible journal. Every individual element is exactly what a real citation would look like.
It just doesn’t exist.
This isn’t lying, because lying requires knowing the truth. The model has no stored list of facts to check against. It has patterns. And “a citation shaped like this usually follows a claim like that” is a pattern it learned very well.
Which is why hallucinations are so convincing. They’re generated by the same machinery as the correct answers, so they arrive in the same confident tone, in the same format, with the same air of authority. There’s no tell.
The rule that follows: the more obscure the fact, the more likely it’s invented. Common knowledge appeared thousands of times in training and is solid. A specific statistic, a page number, a niche case study, a person’s exact job title — thin evidence, strong pattern, high risk.
The training cut-off
Because the model is frozen, it only knows the world as it was when training stopped. That’s the cut-off date, and it’s usually months before you’re using the tool.
Ask about anything after that and one of three things happens: it says it doesn’t know (good), it searches the web (also good, if the tool can), or it confidently describes a world that has moved on (bad, and common).
Anything time-sensitive — prices, current roles, recent news, the latest version of software — check it, or use a tool that searches.
Why it’s brilliant and useless in patches
Lesson 1 mentioned that AI’s abilities are jagged. Here’s the reason.
The model is good at whatever was well-represented in what it read. Explaining a legal concept? Written about endlessly. Drafting a polite refusal? Thousands of examples. It’s excellent at both.
Counting the letters in a word? Almost nobody writes that down, because humans just look. So there’s nothing to learn the pattern from — and the model doesn’t count anyway, it predicts. Same for arithmetic, which is why a system that can write a working program will sometimes fumble a sum.
So don’t reason from “it did something hard, so it can do something easy.” That intuition comes from people, and it doesn’t transfer. The question is never how hard a task is for you. It’s how often it appeared in writing.
What this means in practice
- Common knowledge: trust it. Specific details: check them. The pattern is strength of evidence, not difficulty.
- Restate context in long chats. It’s re-reading, not remembering, and the window has an edge.
- Assume it’s out of date unless the tool is searching.
- Treat confidence as meaningless. Tone tells you nothing about accuracy. There’s no signal in it.
- Give it more to work with. It predicts from what’s in front of it — so what’s in front of it is the whole game. That’s Lesson 4.
Key takeaways
- Models learn by predicting the next word, over and over, until the patterns behind language come with it
- Training ends and the model freezes — it doesn’t learn from your conversations
- “Memory” in a chat is the whole thread being re-sent, and it has a limit
- Hallucinations come from the same machinery as correct answers, which is why they sound identical
- The more obscure the fact, the thinner the evidence and the higher the risk
- Ability tracks how often something was written down, not how difficult it seems
Next: Lesson 3 – The AI Tool Landscape
You know what AI is and how it learns. Next: which tool to actually open, and what each one is genuinely for.