When we think about language models, attention usually takes the spotlight. But if we ask where a model’s knowledge — that the capital of France is Paris, or the syntax of a programming language — is actually stored, the answer lies elsewhere: in the MLP layers.
Two different roles
A transformer block contains two primary components: attention and the MLP (feed-forward network). They perform distinct jobs. Attention routes information: it determines which parts of the text relate to one another and moves context between positions. It does not hold facts itself. The MLP holds the knowledge: it is where facts and patterns are stored.
A standard metaphor applies here: attention is the librarian who knows where to look, while the MLP represents the books themselves. To answer “what is the capital of France”, attention brings the context of “France” and “capital” together, and the MLP retrieves the answer “Paris”.
A key-value kind of memory
Mechanically, the MLP operates as a key-value memory store. The first part of the MLP acts as a set of keys that recognise specific patterns, while the second provides the values retrieved in response. When a specific pattern activates in the input — for instance, a semantic marker for “capital of a country” — the matching key triggers, and the associated value (the stored knowledge) is added to the flow.
Why the MLP is so large
This architecture explains why the MLP accounts for the vast majority of a model’s parameters. In a typical transformer, the MLP layers hold the bulk of the weights, with attention taking a smaller fraction and the remainder allocated to embeddings. The reasoning is structural: storing millions of independent facts — names, dates, relations, and language syntax — requires immense capacity. Routing context is relatively light; storing knowledge is heavy.
Why this matters
This separation of concerns has direct practical consequences. When fine-tuning a model to ingest new facts, the MLP layers must be targeted alongside attention — otherwise, the new knowledge has nowhere to settle. Similarly, when compressing or pruning an architecture, altering the MLP directly degrades the model’s memory.
Putting it together
The core dynamic is straightforward: attention says “look at this”, and the MLP says “here is what I know about it”. A model’s knowledge is distributed across millions of MLP weights, not bound within the attention mechanism. Recognising this separation clarifies both how to approach fine-tuning and why models are so large — most of that parameter count is simply static memory.