# World Models, Simply Explained

#98 Apr 20 RL

https://www.aibutsimple.com/p/world-models-simply-explained

Hello from the AI, but simple team! If you enjoy our content and custom visuals, consider sharing this newsletter with others or [upgrading](http://www.aibutsimple.com/pro?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=world-models-simply-explained) so we can keep doing what we do.

Large Language Models (LLMs) have become a centerpiece in modern AI. Not only are they extremely useful, scalable, and applicable in many industries, but they have also become the most used form of AI for the general public.

While LLMs demonstrate amazing pattern recognition and next-token prediction, they lack a grounded understanding of the physical world, making them unstable when deployed in new environments (e.g., autonomous driving, robotics).

In the pursuit of [Artificial General Intelligence (AGI)](https://newsletter.aibutsimple.com/p/agi-simply-explained), researchers have shifted towards a new type of model to simulate the physical laws and relationships of reality.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/993e6ca4-af33-4940-afba-4933a097e02d/World_Models_Cover__1_.png)

World models are a new type of large-scale model that can understand the real world. At its core, a "world model" gives an artificial intelligence a form of imagination.

- They are foundation models, meaning that they fall in the category of extremely large models trained on massive datasets.
- They can be adapted to a wide range of downstream tasks like text generation, image recognition, etc. Think LLMs like ChatGPT, DeepSeek, or Claude.

Instead of an AI just reacting blindly to what it sees right now, a world model allows the agent to internally simulate, or "dream" about, the future before it makes a decision.

For instance, if an autonomous car is deciding whether to brake or swerve, its world model predicts how the pedestrians, other cars, and physics will react to both choices, allowing it to pick the safest option.

World models can take in text, image, video, and movement to generate videos and navigate realistic physical environments.

### The Foundation Behind World Models

In mathematics, real-world environments are modeled as a Partially Observable Markov Decision Process (POMDP).

This is a technical way of saying that the model, or agent, never gets to see the true (perfect) state of the entire universe.

It only gets noisy, incomplete observations (such as the pixels from a camera or audio waves from a microphone), which we denote as xt.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/db242e20-12f6-4bb8-a208-8284d3680308/Partially_Observable_Markov_Decision_Process__1_.png)

The agent must then make decisions based on this uncertain information by learning what we call the transition dynamics.

In other words, the goal of a world model is to learn how the environment changes from one state to the next, expressed by the below mathematical probability function.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/1c18a00a-c431-4148-9b65-396a54b4856a/image.png)

This is a crucial expression used in model-based RL, and essentially, it asks, "What is the probability of the next state st+1 occurring, given the current state st and the action the AI decides to take at?".

The key idea: world models approximate a latent-space version of this equation.

An important question to ask is, why do we use a latent space representation?

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/e95eaa8a-905c-400f-ae3d-5b7f87bdf1b4/Latent_Space__World_Models__1_.png)

Let’s take the example of using a world model to predict the next frame in a video. You might assume that the agent will attempt to predict the next frame by creating it pixel-by-pixel.

However, predicting every single pixel requires too much computing power and is mostly irrelevant to decision-making.

Instead, world models use an encoder network to compress the high-dimensional observation xt into a smaller mathematical state called a latent state st.

The agent then makes its future predictions entirely inside this compressed latent space.

We’ll see this idea in action in the following section, where we compress car racing frames into a latent space and then use it for efficient simulation.

### The Original “World Models” Paper

Although the idea of world models appeared much earlier, the concept started gaining traction with the groundbreaking 2018 paper “[World Models](https://arxiv.org/pdf/1803.10122?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=world-models-simply-explained)” by David Ha and Jürgen Schmidhuber.

They showed that an AI agent could learn to navigate environments (they tested it in a car racing game) within its own "hallucinated" dream.

- The dream refers to an internal, learned simulation of the environment where the agent can generate and evaluate future states, much like “thinking ahead.”

Their architecture separated the full system into three components:

1. A vision model (V) to compress what it sees,
2. A memory model (M) to predict what happens next,
3. A controller model (C) to make decisions.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/f2db36cd-c283-491a-8e73-6fec884a9a44/World_Model_3_Components__1_.png)

#### The Vision Model (V)

The V model is a Variational Autoencoder (VAE), used to compress high-dimensional visual frames into a low-dimensional, continuous latent vector (zt).

- Compression is used to make future state prediction more efficient, since operating directly on raw pixels is computationally expensive.

This acts as the agent's spatial representation of the world at a single moment in time, stripping away noise and other unimportant details.

#### The Memory Model (M)

Since the world is a sequence of events, the agent needs to track time and make predictions for future states.

To do so, we use the M model, a Mixture Density Network Recurrent Neural Network (MDN-RNN).

It takes the current latent vector zt and the agent's action at and uses its internal hidden state ht (its “memory” of previous states) to predict the probability distribution of the next possible latent states.

Mathematically, it models the below probability function.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/c3fd0c45-f102-44bd-a9b7-9714146eea4f/image.png)

The use of a mixture density network allows the model to predict multiple possible futures in environments with high uncertainty.

The V and M models combined make up the “world model” portion of the full model, since together, they model the environment and the latent dynamics.

#### The Controller Model (C)

The C model maps the concatenated information from the vision and memory modules (zt, ht) directly to a physical action. It is essentially the "brain" that makes decisions.

Because the V and M models handle most of the understanding of space and time, the C model can be incredibly small. In fact, the C module is often a simple, single-layer linear network.

During training, the system loops the M model’s predictions back into itself, predicting the next state, zt+1. Then, the C model decides on an action based on that prediction, and the M module uses that action to predict zt+2.

This creates a fully internal simulation loop; the system is feeding its own predictions back into itself to generate the next step.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/0e68bec3-ec21-4d23-b243-1615595d5edc/Controller_Model_Simulation_World_Models__1_.png)

As the C module is trained entirely inside this mathematical "dream" representation, it can simulate millions of rounds of a game (e.g., car racing) in a fraction of the time it would take to render the actual game engine.

Since the agent can practice within the internal simulation (rather than the actual game environment), it can generate large amounts of experience at a fraction of the cost, allowing it learn much faster and more efficiently.

This is the idea that laid the foundation for the complex world models we see today.

---

### [Level Up Your Career With AI, But Simple PRO](https://www.aibutsimple.com/pro?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=world-models-simply-explained)

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/281697a8-c915-435c-a68b-43374b671698/aibutsimple_academy.png)

Supporting us by purchasing a PRO membership will get you:

- Exclusive Hands-on Tutorials With Code
- Real-World Interview Questions and Cases (From Real Companies)
- Full Math Explanations and Beautiful Visualizations
- Ad-Free Reading for an Uninterrupted Learning Experience

[Break Into AI With PRO](https://www.aibutsimple.com/pro?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=world-models-simply-explained)

---

### Popular World Model Architectures

#### DreamerV3

[DreamerV3](https://arxiv.org/abs/2301.04104?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=world-models-simply-explained) (Hafner et al., 2023) is one of the most successful (and popular) world models for robotics and complex video games, being the first model to mine diamonds in Minecraft from scratch. It works using a Recurrent State-Space Model (RSSM).

To handle unpredictable environments and to provide a wide downstream applicability, DreamerV3 splits its latent state (st) into 2 parts, shown below.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/a8e83908-ea8a-4d8a-8f94-ffe34e0dc346/image.png)

Here, ht is a deterministic memory (things the model is certain of, based on the past), and zt is a stochastic, random variable (unpredictable events that might happen).

DreamerV3 trains its decision-making policy by rolling out thousands of imagined future states in the latent space. Just like the 2018 world model, it learns without having to practice in the real world.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/e189b811-0876-4fe5-9a10-9b4d73ba6c6b/Dreamer_World_Models__1_.png)

#### Dreamer 4

[Dreamer 4](https://arxiv.org/abs/2509.24527?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=world-models-simply-explained) (Hafner et al., 2025) was released as an update to the Dreamer lineage.

Dreamer 4 shifts the focus from online reinforcement learning (where the agent learns by actively playing the game) to an entirely offline setting.

It learns to solve complex, “long-horizon” tasks, such as mining diamonds in Minecraft, purely by observing pre-collected datasets without ever interacting with the live environment during training.

To achieve this, Dreamer 4 abandons the RSSM used in DreamerV1 to DreamerV3 and adopts a highly scalable Diffusion Transformer (DiT) backbone. Specifically, the world model works as a conditional latent video DiT.

Dreamer 4 consists of an interactive dynamics model (the transformer) and a causal tokenizer.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/504f66c9-a6e2-447d-80a4-0757d47dbddb/Dreamer_World_Models__2_.png)

Dreamer 4 Architecture From Paper

The transition dynamics (probability function) in Dreamer 4 are built to predict the next latent state zt based on a context window of previous latent states zt-c:t-1 and the current action at, shown below.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/5a8292ed-b14d-4695-a0b9-f7eeae72b090/image.png)

By using a diffusion process in the latent space, DreamerV4 reduces long-horizon prediction errors, allowing the agent to “dream” of extended and accurate video sequences of the future.

#### JEPA, JEPA-Based World Models

The Joint-Embedding Predictive Architecture (JEPA) predicts abstract representations, not pixels. It was pioneered by Yann Lecun at Meta, and it’s a useful building block of world models.

- It’s important to note that JEPA itself is not a world model. If you want an agent to drive a car, we take the JEPA architecture and build a planner around it to pick actions based off JEPA outputs.

Instead of a decoder, JEPA uses a second encoder to process the future observation/input into a target abstract representation. This is the representation that differentiates JEPA from other architectures.

It’s crucial to note that JEPA does not use any decoders in its core objective. The team argues that reconstructing pixels is a waste of compute and that the subtle details are not necessary to learn the general dynamics of the environment.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/8bd491b0-801c-4c16-a895-a7fa121ed82c/JEPA__World_Model.png)

However, predicting abstract representations instead of pixels creates a mathematical problem called "representation collapse."

- The agent could cheat by simply predicting constant values identical to the target, resulting in zero loss but learning nothing.

To fix this, JEPA uses Variance-Invariance-Covariance Regularization (VICReg).

VICREG modifies the loss function by adding a variance and covariance regularization term within the objective. The variance term prevents collapse to constant values, while the covariance term reduces redundancy, resulting in useful feature learning.

Recently, researchers have taken the JEPA component and have built [JEPA-based world models](https://arxiv.org/abs/2601.00844?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=world-models-simply-explained) around it. Examples include [V-JEPA 2](https://arxiv.org/abs/2506.09985?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=world-models-simply-explained) and [C-JEPA](https://arxiv.org/abs/2602.11389?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=world-models-simply-explained).

We recently wrote an article about [VL-JEPA](https://arxiv.org/abs/2512.10942?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=world-models-simply-explained), a vision-language JEPA-based model developed by meta’s FAIR team. You can learn more about it here:

- [VL-JEPA, Simply Explained](https://newsletter.aibutsimple.com/p/vision-language-models-vl-jepa-simply-explained)

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/baecfd0c-1329-4018-913e-85f192805d45/Copy_of_AI__But_Simple_Visuals_March_26__6_.png)

#### DRAMA (Mamba-Based)

[DRAMA (Wang et al., 2024)](https://arxiv.org/abs/2410.08893?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=world-models-simply-explained) is a world model that uses the State Space Model (SSM) architecture.

It solves one of the key issues in transformer-based world models: how compute scales quadratically (O(N2 )) with respect to the input length.

New models like DRAMA are built on State Space Models (SSMs) like Mamba. Instead of self-attention, Mamba uses continuous differential equations:

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/83001e26-a4ca-4807-b48c-f2154fe44281/Explaining_SSM_Equations.png)

The SSM state equation tracks how the internal state (h) evolves based on its own matrix A and how the input data influences it (B). The output equation tracks how the current state (h) transitions into the output and how the input (x) affects the output.

As the computations can be processed in linear time (O(N)), the world model can theoretically process the past and imagine the future quicker than their transformer counterparts.

To learn more about SSMs, we covered the basics of these models in a previous issue, found here:

- [Mamba and State Space Models, Simply Explained](https://newsletter.aibutsimple.com/p/mamba-and-state-space-models-simply-explained)

#### What’s Next For World Models?

By giving models the ability to simulate real environments, world models are unlocking the next paradigm of deep learning.

World models are currently used in applications such as generating custom synthetic data or downstream AI models for training robots and autonomous vehicles.

Video generation world models are also popular recently. These models treat the physical world the same way text models treat paragraphs. Examples include [OpenAI’s Sora](https://openai.com/sora/?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=world-models-simply-explained) and [Deepmind’s Genie](https://deepmind.google/models/genie/?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=world-models-simply-explained).

As world models become increasingly popular in the coming years, they become the vital bridge between the pure next-token prediction of LLMs and the physical reality we live in.

---

Here’s a special thanks to our [biggest supporters](https://www.aibutsimple.com/upgrade?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=world-models-simply-explained):

- Sushant Waidande
- Sai Krishna Pashapu
- Bob Boncella
- Demet Yazici

---

If you enjoy our content, consider supporting us so we can keep doing what we do. Please share this with a friend!

Want to reach 50000+ ML engineers? Let’s work together:

[WORK WITH US 📈](https://tally.so/r/A7Dgdk?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=world-models-simply-explained)

Feedback, inquiries? Send us an email at [edwin@aibutsimple.com](mailto:edwin@aibutsimple.com).

---

That’s it for this week’s issue of AI, but simple. See you next week!

—AI, but simple team