Issue #111 LLMs Edwin Dong
Foundation Models, Simply Explained
Free editionPRO edition
This week’s PRO issue includes a Colab notebook testing various vision, language, and CLIP models with practice exercises and solutions.
Please do read the full article before attempting the code tutorial.
As always, the code in this issue can be executed on a normal home PC and does not require a large amount of VRAM.
The monthly interview question bank can be found here. It has been updated for July!
These are industry standard questions that are tested across major tech companies, AI research firms, and more. The questions are sorted by difficulty and topic.
Check back frequently for more questions, and let us know what type of questions you would like to see next!
A couple years ago, most models were experts at one use case. A machine translation model translated. A spam filter filtered spam. Each one was either trained or fine-tuned on its own dataset for its own narrow task.
Foundation models were a key directional change that breaks the “one use case” pattern. The term was defined in a 2021 Stanford report with over 100 researchers in collaboration.
A foundation model is a model trained on broad data at scale that can be adapted to a wide range of downstream tasks.
Train one huge model once on a massive corpus of mostly unlabeled data, then adapt that model to translation, spam detection, captioning, and hundreds of other use cases it was never directly trained on.
Examples of foundation models we know and love today include:
LLMs (language or vision-language) such as GPT, Llama, Claude, Gemini
Vision models such as DINOv3, SAM 3
Image generation models like Stable Diffusion 3, FLUX.1, Imagen 4
Video generation models such as Sora, Veo 3, Cosmos Transfer
World models like NVIDIA Cosmos World Foundation Models, Genie 2
And much, much more.
Today, we’ll break foundation models down and explore the research problems that decide how performant, how fast, and how cheap these models actually are.
This edition is for PRO members.
