Issue #109 LLMs Edwin Dong, Anurag Shinde
Diffusion Language Models (DLMs), Simply Explained
Free editionPRO edition
This week’s PRO issue includes a Colab notebook with practice exercises and solutions for diffusion generation techniques. We’ll dive deep into diffusion language models, and see how they work with interactive graphs.
Please do read the full article before attempting the code tutorial.
As always, the code in this issue can be executed on a normal home PC and does not require a large amount of VRAM.
The monthly interview question bank can be found here.
These are industry standard questions that are tested across major tech companies, AI research firms, and more. The questions are sorted by difficulty and topic.
Check back frequently for more questions, and let us know what type of questions you would like to see next!
Many language models you use today work autoregressively, generating tokens left to right given the context of what came before.
LLMs like GPT, LLaMA, and Claude all mostly follow this structure.
This idea is so deeply baked into how we build these models that most researchers also treat it as a given.
A team from Renmin University and Ant Group decided to ask a simple question: Does it always have to be this way?
Can language models instead be guided by the stochastic process of diffusion for generating text, much like for generating high-fidelity images and video?
Their answer is in the form of a model called LLaDA (Nie et al., 2025), and it is a new interesting direction.
This edition is for PRO members.
