# Fine-Tuning for LLMs, Simply Explained

#87 Feb 2 Training

https://www.aibutsimple.com/p/fine-tuning-for-llms-simply-explained

Think about the last time that you used an LLM. The usual ChatGPT 5.2 model or Claude Sonnet 4.5 would suffice for most tasks: finding a quick answer to a burning question, feedback on a random idea you had, or asking for homework help.

The good thing is that these proprietary models have seen such a large volume of data while they were being trained that they have a decent amount of knowledge in most domains.

- In this form, they most certainly have breadth of knowledge but may lack in depth.

Specialization is key, and that’s how fine-tuning fits into the picture. Organizations, companies, and legal bodies may need the assistance of an LLM in a highly specialized manner.

For example, a hospital that is using an LLM to help write tedious and lengthy paperwork, or a legal office that is analyzing hundreds of documents.

Fine-tuning an LLM allows you to harness specific capabilities and tailor them to the task at hand.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/98f54fd3-3eea-49fa-a628-5713fa78fc5b/LLM_Fine-Tuning_COVER.png)

In concept, this sounds very simple. Just like how an LLM was trained with billions of data points, you may expect to “re-train” this LLM using your own custom dataset/classification.

With this, you can imagine the process as having an LLM trained on “the basics” and then “teaching” it the skills you want it to have.

The process we have outlined is almost what LLM fine-tuning is, but a lot happens under the hood mathematically to ensure that these skills are “learned.”

In implementation, one will have to understand how gradient descent on millions of parameters will need to be handled.

---

### [Level Up Your Career With AI, But Simple PRO](https://www.aibutsimple.com/upgrade?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=fine-tuning-for-llms-simply-explained)

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/281697a8-c915-435c-a68b-43374b671698/aibutsimple_academy.png)

Supporting us by purchasing a PRO membership will get you:

- Exclusive Hands-on Tutorials With Code
- Full Math Explanations and Beautiful Visualizations
- Ad-Free Reading for an Uninterrupted Learning Experience

[Break into AI or sharpen your edge by joining today!](https://www.aibutsimple.com/upgrade?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=fine-tuning-for-llms-simply-explained)

---

### Under the Hood

For one, we should first start with a model that is pre-trained, or in other words, a model that has already been trained on a text dataset.

This model understands some basic semantic language dynamics and general knowledge. For example, a very basic text transformer would fit the description.

With this pre-trained model, you can use your desired dataset to begin training by splitting it into two parts: a training set and a testing set.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/0cadc10f-7a9c-41d4-b9b1-7b3fb2d2fb78/LLM_Fine-Tuning.png)

For instance, if you’re training a customer-service chatbot, you would let it process chat transcripts from real conversations (or synthetically made conversations), while also guiding it to know what types of conversations are “good” and “bad.”

This element of classification is very important for the model. It allows it to learn certain groups of characteristics and ignore others.

In that customer-service chatbot, you’ll begin with the training set and may label a conversation where the representative is rude as a negative interaction, whereas one that resolves a customer’s issue thoroughly would be a positive interaction.

During the training set, the model will also try and predict how conversations will go for positive interactions and then compare how far off it was. That measurement is called loss.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/9015c5ef-5ab2-4a58-84be-20bcb4473ccd/LLM_Fine-Tuning__1_.png)

Using the value of the loss, the model will adjust its parameters to try and reduce the loss as much as it can.

Therefore, the model will not only need to hold the value of each of these millions of parameters but also the change, or gradient (the vector that contains the direction and magnitude of change for each parameter).

When the model iterates backwards through each layer and adjusts the parameters, a familiar backpropagation occurs.

### A Challenge Appears

In models that have up to billions of parameters, a huge problem appears: memory/storage.

Remember, we have to store each of the actual parameter values (float32 or float64) and their respective gradients, along with something called an optimizer state.

The optimizer state contains information regarding the parameters’ momentum and variance, which create a weighted average of the squared gradients.

Let’s go through an example with numbers. Consider fine-tuning a relatively modest model like Llama-2-7B, which has 7 billion parameters. When storing parameters as 32-bit floats:

- Model weights: 7 billion parameters x 4 bytes = 28 gigabytes (GB).
- Gradients: Matches the amount you need for weights = 28 GB.
- Optimizer state: For the Adam optimizer (a common option), this stores two values per parameter, momentum and variance = 56 GB.

The total base requirement comes out to about 112 GB.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/8fda1868-08b8-4448-aa13-31e1256e3f12/LLM_Fine-Tuning__2_.png)

As you can see, this requires quite a lot of memory, which even a standard NVIDIA A100 GPU, a GPU made for huge data centers, could not provide all at once. The A100 only has about 80GB of memory, which shows that fine-tuning this normally is not practical.

You may be wondering why we are using the optimizer state. If we completely discarded that, we could train our model viably. The momentum and variance help stabilize the training process.

- Momentum tracks recent gradients changing over time, which gives the model an idea of which direction is the appropriate one.
- Variance tracks how much gradients fluctuate, which helps the model make appropriate changes to its predictions.

Take the scenario of adjusting a thermostat. When temperature readings change wildly (high variance), you should make small adjustments to how you’re moving it. If the temperature is not moving as much (low variance), you can take a “larger” step without fearing a drastic change.

Returning to our discussion on impractical memory storage, there have been optimizations that are used to mitigate the costs. Of these, a popular solution is LoRA (Low-Rank Adaption), which is part of a larger family called PEFT (Parameter-Efficient Fine-Tuning).

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/5c2333f8-f34f-4e58-a373-0632c0bf56d3/LoRA.png)

PEFT optimizations aim to adjust a small subset of relevant parameters, which cuts down the memory requirement substantially.

Another method of fine-tuning can also be applied here: Instead of changing the model weights, you can add smaller additional layers between the existing ones. You only train these new layers and keep the original model frozen. This may not work as well, however.

### Model Considerations

Beyond the technical constraints, it’s important to talk about how we should plan to have the model undergo the actual process of fine-tuning.

Interestingly, fine-tuning only requires a fraction of the amount of data that would be needed to train a model from scratch. The fine-tuning stage might only need hundreds or thousands of examples, as compared to the billions of tokens that are processed in pre-training.

The reason for this is that the model already knows a baseline for the English language and is just utilizing pattern recognition to improve its abilities to fulfill a certain task.

An important distinction between the pre-training and fine-tuning stages is the fact that fine-tuning relies on the adage of “quality over quantity,” whereas pre-training benefits from sheer volume.

We must also discuss the existence of the learning rate, which can be thought of as a hyperparameter that governs how the model takes risk.

We’ve talked about the optimizer state playing a role in how much of a change in parameters that the model may tolerate, but the learning rate influences how much a model changes its overall predictions.

If this learning rate is high, the model runs the risk of overshooting an optimal solution, but if it’s too low, it may expend resources in slowly trying to find a performance sweet spot. Typically, a learning rate of the magnitude of -5 or -6 is used for fine-tuning a model.

Each pass through the dataset, where the model learns with a combination of influence from its learning rate, optimizer state, and dataset size, is called an epoch.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/e85b523d-3eac-4be1-a9c8-ce5882b9e73b/LLM_Fine-Tuning__3_.png)

Fine-tuning may require up to ten epochs, whereas pre-training will require hundreds of epochs. Like picking a good-ranged learning rate, the number of epochs will matter as well to avoid overfitting.

Overfitting is a phenomenon that describes when a model is able to score high when faced with data it has been trained on but fails to generalize results when given unfamiliar data.

The testing set we mentioned earlier? This is where that comes into play, as we can avoid overfitting when we see the model’s score on unfamiliar data starting to plateau.

### The Bottom Line

In this article, we’ve taken a deep dive into everything we need to know when it comes to fine-tuning a pre-trained LLM.

In our Pro-version article for this week, we take you through the process of actually performing this to your own model, allowing you to it to whatever domain you would like.

Fine-tuning is the process that allows LLMs to be much more than just untapped knowledge banks. It’s a hands-on process that can reveal a lot about model tendencies when it learns traits and abilities and is a cornerstone of modern industry machine learning.

---

Here’s a special thanks to our [biggest supporters](https://www.aibutsimple.com/upgrade?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=fine-tuning-for-llms-simply-explained):

- Sushant Waidande
- Sai Krishna Pashapu
- Bob Boncella
- Demet Yazici

---

If you enjoy our content, consider supporting us so we can keep doing what we do. Please share this with a friend!

Want to reach 50000+ ML engineers? Let’s work together:

[WORK WITH US 📈](https://www.passionfroot.me/aibutsimple?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=fine-tuning-for-llms-simply-explained)

Feedback, inquiries? Send us an email at [edwin@aibutsimple.com](mailto:edwin@aibutsimple.com).

If you like, you can also donate to help our team push out better newsletters every week!

[Support us ☕](https://ko-fi.com/aibutsimple?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=fine-tuning-for-llms-simply-explained)

---

That’s it for this week’s issue of AI, but simple. See you next week!

—AI, but simple team