1.6 — The generative era & hallucination 🎨

In 2017, a new architecture called the Transformer (paper “Attention Is All You Need”) changed everything. Before, AI analyzed (“this is a photo of a cat”). Now it creates: text, images, code. This is the birth of the Large Language Models (LLMs) like ChatGPT, Claude, or Gemini.

🔍 The secret in 30 seconds: how an LLM “writes.” It doesn’t “think through” its sentence. It predicts the most probable word (or piece of a word), adds that word, then starts again — over and over. Like a super-powered autocomplete, trained on billions of texts. Example: after “The cat drinks…”, it computes that “milk” is very likely and suggests it.

⚠️ Direct consequence — hallucination. Since the model aims for the plausible and not the true, it can invent a wrong answer with total confidence. This isn’t an occasional bug, it’s structural. Much of this training (prompting, tools, guardrails) exists precisely to tame this behavior.

🟢 In plain words — an LLM is a giant autocomplete: it guesses the most probable next word, over and over. It doesn’t know whether it’s telling the truth — it says what sounds right.

⚠️ Common mistake — assuming an AI answer must be true because it’s well written and self-assured. A confident tone proves nothing. The right reflex: verify the important facts (dates, figures, names) against another source.

🔎 Going further

The model doesn’t reason word by word but by “tokens” (pieces of words). For each token it computes a probability across tens of thousands of candidates and picks one. That’s why the same prompt can give two slightly different answers: there’s an element of controlled randomness.

Era	Core idea	What the AI does
Expert systems (70s-80s)	Humans write the rules	Apply IF…THEN
Deep Learning (2010s)	The machine learns by example	Analyzes / recognizes
Generative (since 2017)	Predict the next item in a sequence	Creates content

✅ In short

• The Transformer (2017) architecture moved AI from analysis to creation.
• An LLM writes by predicting the most probable next word, in a loop.
• Hallucination (a wrong but confident answer) is a structural consequence, not a bug.