Issue #111 LLMs Edwin Dong
Foundation Models, Simply Explained
A couple years ago, most models were experts at one use case. A machine translation model translated. A spam filter filtered spam. Each one was either trained or fine-tuned on its own dataset for its own narrow task.
Foundation models were a key directional change that breaks the “one use case” pattern. The term was defined in a 2021 Stanford report with over 100 researchers in collaboration.
A foundation model is a model trained on broad data at scale that can be adapted to a wide range of downstream tasks.
Train one huge model once on a massive corpus of mostly unlabeled data, then adapt that model to translation, spam detection, captioning, and hundreds of other use cases it was never directly trained on.
Examples of foundation models we know and love today include:
LLMs (language or vision-language) such as GPT, Llama, Claude, Gemini
Vision models such as DINOv3, SAM 3
Image generation models like Stable Diffusion 3, FLUX.1, Imagen 4
Video generation models such as Sora, Veo 3, Cosmos Transfer
World models like NVIDIA Cosmos World Foundation Models, Genie 2
And much, much more.
Today, we’ll break foundation models down and explore the research problems that decide how performant, how fast, and how cheap these models actually are.
