A skeptic’s honest question

You have probably heard a version of this before. A colleague forwards an article β€” β€œAI is transforming everything” β€” and you feel a flicker of dΓ©jΓ  vu. Didn’t we do this in the nineties? Expert systems. The knowledge revolution. Then the AI winter. Then again in the 2010s. And the winter again. When someone says things are genuinely different now, the healthy response is: show me.

That is what this post does. This is a plain account of three eras in the field, each defined by a single question: who does the learning?

Era one β€” humans write every rule

The earliest practical AI systems were expert systems. The premise was straightforward: take a human expert (a doctor, a loan officer, a factory inspector) and interview them. Write down every rule they know. Then codify those rules into software: if the patient has these three symptoms and not this fourth one, suspect diagnosis X.

This works, provided you stay within the narrow boundaries you drew. The problem is that real situations do not respect boundaries. A loan applicant rarely presents the exact criteria in your rulebook. A disease presents atypically. Every exception requires a human to review the logic, update the rules, and redeploy. The system does not learn from what it observes; it only executes what a human thought to write down.

Who does the learning: the human expert. The machine just executes.

The brittleness was real. As the world changes, someone has to rewrite the rules. It scales poorly, and it fails quietly at the edges of its rulebook.

Era two β€” machines learn patterns, but from features a human chose

The next wave β€” classical machine learning β€” shifted the workload. Instead of writing rules by hand, you give the machine labelled examples: here are ten thousand emails, each marked spam or not spam. The machine finds the statistical patterns on its own.

This is genuine progress. The machine discovers patterns a human would not think to encode. It handles variation better, and it can update as new examples arrive.

But there is a catch. To apply machine learning, a human still has to decide which features (measurable signals from the data) are worth analysing. For spam, this might mean: does it contain certain words? How long is it? What time was it sent? A domain expert curates the feature set. The machine then learns which combination of those chosen signals predicts the label.

There is one more constraint: you train one model per task. A spam classifier learns only about spam. It cannot help you with a translation or a credit decision. Every new problem starts from scratch.

Who does the learning: the machine learns patterns β€” but from a menu of signals a human curated. It learns narrowly: one model, one task.

Era three β€” one model learns broadly; a human just adapts it

This is the shift defining the current wave. A foundation model is pre-trained on a vast, general corpus β€” text from across the web, books, code, and scientific papers. It learns broadly: language, world knowledge, and reasoning patterns. Because this pre-training requires enormous amounts of compute and data, it happens rarely, executed by labs with the necessary resources.

What emerges is a general-purpose base. To put it to work on a specific task, you adapt it β€” sometimes with a handful of examples, sometimes just a well-written instruction. You do not retrain from scratch. You steer a system that already knows a great deal.

This is sometimes called a β€œfoundation model” or a β€œlarge language model” (when the modality is text). The term that matters is general-purpose: one model, many tasks.

Who does the learning: the machine learns broadly from vast data. The human’s job is now adaptation β€” pointing a capable model at a problem rather than building a new model for it.

What unlocked it β€” scale and a new architecture

Two things converged to make era three possible.

First, scale. Dramatically more training data and compute became available through the 2010s. Researchers at several labs began noticing something worth citing carefully: according to findings published in the machine-learning literature, certain abilities appeared at large scales that were not present at smaller ones. This phenomenon β€” sometimes called β€œemergent abilities” β€” is a genuine finding, though the research community continues to debate what it precisely means and how reliably it holds. It is a real observation, not marketing.

Second, a new architecture called the transformer (introduced in a 2017 paper titled β€œAttention Is All You Need”). Without getting into the maths: a transformer is a design for a neural network that is especially good at finding relationships across long stretches of text β€” not just the words immediately next to each other. This made it practical to train models on much longer contexts and much larger datasets than earlier designs allowed. It became the foundation for the models deployed today.

Scale and the transformer together made the leap to era three feasible.

Why the current wave is not the last hype cycle

In the first two eras, a new model for a new task could go out of date quickly, or fail at the edges of its training. The promise of era three is different in kind: generality and adaptability. A foundation model that has learned broadly is harder to make obsolete with a single counter-example, because it was never limited to a single narrow task.

That said, honesty requires a caveat: the current wave has real limits. Foundation models still hallucinate β€” they produce confident-sounding errors. They have knowledge cut-offs. They struggle with precise arithmetic and strict determinism. None of that is resolved by simply saying β€œera three”. Those limits are worth understanding β€” and the next post in this series addresses them directly.

One thing the current wave is not: artificial general intelligence. Nothing deployed today generalises the way a human does across arbitrary new situations with no prior exposure. The era-three claim is narrower and more defensible: one model, many tasks, general-purpose within a domain β€” not β€œhuman-level reasoning across all of life”. Be sceptical of sources that slide from β€œpowerful language model” to β€œAGI” without marking the gap.

A map for the rest of the series

If the three eras explain why foundation models are the dominant paradigm now, the companion posts detail what they actually are and how they work:

  • AI is not one thing β€” a walkable map from β€œAI” down to LLMs, chatbots, and agents, and why the model is not the product
  • From LLMs to agents β€” what gets added on top of a foundation model to make it take action in the world

The shift from era one to era three did not remove the need to understand what you are working with. It made it more important.