Modern AI Is a Representation Update Machine

Simple parts, repeated at scale, shaped by data.

The strange thing about modern AI is not how complicated it is. It is how simple it looks once you get close enough.

Self-attention. Cross-attention. Projection heads. Normalization. Residual connections. MLPs. The same small set of parts appears again and again, under different names, across language models, image generators, video models, and multimodal systems.

This can feel almost disappointing at first. Shouldn’t the most advanced technology in the world have a more mysterious core?

But the mystery is not in the individual operation. The mystery is in what happens when a simple operation is repeated at scale, shaped by data, and forced by an objective to build useful representations.

Modern AI is not best understood as a pile of tricks. It is a learned state-update system.

At the center is a simple loop:

take a state
read from memory
transform the state
stabilize it
repeat

The state is the model’s current internal representation. In a language model, it may represent a sequence of text tokens. In a diffusion transformer, it may represent noisy image or video patches. In a multimodal model, it may mix text, image, audio, or other signals.

The details change. The loop remains.

The State

The first useful question to ask about any modern model is simple:

What is being updated?

For a language model, the state is a sequence of token representations. Each token carries some partial view of the text so far. Layer by layer, the model updates these representations until they become useful for predicting the next token.

For a diffusion model, the state is different. It may start as noise. Each step updates that noisy state toward something that looks more like an image or video from the training distribution.

For a multimodal model, the state might be text reading from an image, or image tokens reading from text, or a shared representation where different modalities can interact.

So the model is not “thinking” in words, pixels, or frames directly. It is updating vectors that stand in for them. The raw input is converted into a space where the model can move, compare, combine, and reshape information.

That is the first mental model:

AI models work by moving representations through a learned space.

Attention Is Memory Access

Attention sounds more magical than it is.

At its core, attention lets one representation ask:

What should I read right now?

A query looks at a set of keys, decides which ones are relevant, and reads from the corresponding values. The output is a weighted mixture of information.

Self-attention means the model reads from itself. Each token can look at other tokens in the same state. This lets information move across a sentence, an image, or a video.

Cross-attention means the model reads from a condition. A video latent can read from a text prompt. An image token can read from a reference image. A generation process can read from a style, a camera path, a mask, or another signal.

Retrieval is the same idea extended outward. The model reads from external documents or memory. KV cache is also a kind of memory: the model reuses past keys and values so it does not have to recompute the whole past again.

The names differ, but the pattern is stable:

self-attention reads from the current state
cross-attention reads from a condition
retrieval reads from external memory
KV cache reads from the past

Attention is not the whole intelligence of the model. It is a way to route information.

MLPs Do The Local Work

People often talk as if attention is the whole story. It is not.

If attention is communication, the MLP is local computation.

Attention decides what information should meet. The MLP decides what to do with it once it arrives. It transforms each token’s representation, activates features, suppresses others, and reshapes the local state.

A simple way to see the division is:

attention moves information between tokens
MLPs transform information inside tokens

Normalization keeps these updates stable. Residual connections keep the state continuous across layers.

Again, none of these pieces is exotic by itself.

The model becomes powerful because the pieces are composed into a repeated update:

state = state + attention(...)
state = state + MLP(...)

Layer after layer, the model writes small changes into the same stream of representation.

How Representations Enter And Leave The Loop

There is another set of terms that often appears near the update loop: heads, projections, and adapters.

They are related, but they are not the same kind of thing.

A head is usually a readout. It turns the internal state into a task output. In a language model, the final head maps hidden states to a vocabulary distribution. In a diffusion model, a prediction head may map hidden states to a noise or velocity estimate. In a classifier, it maps the state to labels.

A projection changes the shape or coordinate system of a representation. It lets one space talk to another: text features into the width expected by an image model, image features into a shared embedding space, or hidden states into the query, key, and value spaces used by attention.

An adapter is different. It is a small trainable path added to an existing model. It can bend a frozen backbone toward a new task, connect a new modality, or specialize behavior without retraining the whole system.

So the core loop still remains:

update the state

But readouts, projections, and adapters decide how representations enter the loop, how they leave it, and how the loop can be redirected without rebuilding the whole model.

The Objective Shapes The Machine

The architecture gives the model a way to update representations. But the objective decides what those updates must become good at.

A language model is trained to predict the next token. That sounds narrow. But to do it well, the model is pressured to learn grammar, facts, style, causality, plans, social patterns, code structure, and traces of reasoning.

A diffusion model is trained to denoise. That also sounds narrow. But to denoise well, it must learn the shape of natural images, objects, lighting, motion, texture, composition, and the relation between prompts and visual structure.

CLIP-style models are trained to align images and text. Preference models and RLHF-style methods push raw capability toward outputs humans prefer.

The important point is this:

The primitive operation is simple.
The learned geometry is not.

The model is not just executing attention and matrix multiplication. It is executing attention and matrix multiplication after training has carved meaning into the weights.

That is where the intelligence lives: not in a single operation, but in the representation space shaped by the training pressure.

One Model For Many Models

This gives a compact way to understand many modern systems.

A language model updates a sequence toward a plausible continuation.

A diffusion transformer updates noise toward the data manifold.

A multimodal model updates one representation while reading from another.

An agentic system adds tools and memory, then lets the model update its state using observations from the outside world.

Different systems vary along a few fundamental dimensions:

What is the state?
What are the tokens?
Where does memory come from?
What updates the state?
What objective shaped the updates?
What controls the generation?

Once you can answer these questions, a new model becomes less mysterious. You do not need to memorize every architecture name. You can ask what kind of state it keeps, what information it can read, and what training pressure shaped its update function.

The same mental model transfers across domains.

Text generation is not separate from image generation in the deepest sense. Both are learned processes for moving a representation toward a target distribution. The tokens are different. The objectives are different. The manifold is different. But the underlying pattern is shared.

Simple Does Not Mean Easy

There is a trap here.

Once the pieces look simple, it is tempting to conclude that modern AI itself is simple. That is wrong.

Matrix multiplication is simple. Training a frontier model stably is not. Attention is simple. Making it work across language, images, video, tools, memory, and human preference is not. A loss function can be written in a line. Choosing the right data mixture, scale, schedule, architecture details, and inference strategy is a serious engineering problem.

The primitives are simple. The system is not.

A useful final compression is this:

Modern AI is a representation update machine.
It keeps a state, reads from memory, applies learned updates, and is shaped by an objective.

This is why the same building blocks keep reappearing. They are general enough to update many kinds of representations. Text, images, video, audio, actions, and tool outputs can all be turned into tokens. Once they are tokens, the model can route information, transform it, and learn the geometry of the domain.

Modern AI is simple in the way physics is simple: a few rules, repeated at scale, produce behavior that no single rule seems to contain.