LoRA (Low Rank Adaptation) Fine-Tuning for LLMs

Large Language Models (LLMs) have proven incredibly useful for a variety of applications in our day-to-day lives. But to create and train a new model from scratch becomes very expensive as models scale in size and often requires supercomputers to do so.

To tackle this, rather than build a new model every time we’re faced with a new problem, we can tweak an existing one. This technique is called fine-tuning, and it adapts a model to new tasks at a fraction of the cost of full training.

Fine-tuning an existing model allows us to change only certain features to our necessity. Although fine-tuning is a comparatively more efficient way to make practical models, it can also become far too computationally expensive when dealing with large-scale models that house hundreds of billions of parameters.

This leads us to Low-Rank Adaptation (LoRA), a popular technique that makes fine-tuning even more efficient. It was introduced in a 2021 paper titled “LoRA: Low-Rank Adaptation of Large Language Models.”

Not only does LoRA adapt an already existing model, it saves on memory by ignoring a majority of its parameters while doing so. Instead of updating every weight in the network, LoRA introduces small adapter layers to the original model.

During training, only these new adapter layers are updated, whilst the rest of the model remains untouched. For instance, take a standard neural network. Standard fine-tuning will update all nodes, while LoRA only updates a fraction of the nodes.

The diagram above highlights which nodes are updated in standard fine-tuning versus a LoRA implementation below.

The top neural network is majorly unchanged but has a new additional layer right at the end. This is the effective implementation of LoRA, where most of a network remains unchanged, and only minor and specific adaptations are required to adapt the purpose of the network for something else.