# Alignment In Transformers, Simply Explained

#66 Sep 1, 2025 Safety

https://www.aibutsimple.com/p/alignment-in-transformers-simply-explained

Transformer models are the most widely used architecture in AI at the moment. Some of its most popular uses are Natural Language Processing (NLP) tasks and multimodal tasks such as Image Captioning.

Transformers are one of the key architectures that tackle the problem of alignment.

Well, what exactly is alignment? In NLP, if we look at language translation, the difference in order and context of words forms a sentence’s meaning—we must figure out which input words correspond to which target words.

- This process is essentially what alignment is: figuring out which part(s) of the input corresponds to which part(s) of the output.

In traditional models, alignment had to be solved as a separate task, but with the arrival of transformers and their “self-attention” mechanism, alignment became integrated within the model.

Because of this “self-attention,” transformers automatically learn the relationship between different parts of an input, such as the relationship between words in a sentence. Alignment is no longer separate but learned as part of training.

To better understand what alignment is and its math intuition, we’ll go through the basic mathematical framework of the alignment problem:

Given an input sequence X:

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/6c4028bb-8f2b-46fb-baa5-ebef20a116b6/image.png)

And a target sequence Y:

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/87f2b169-c1bd-412c-8a68-93bfc98297e1/image.png)

The alignment problem seeks to find a mapping that matches elements in the target sequence Y to elements in the input sequence X. This is typically done via an alignment mapping function:

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/288441a3-de83-47e4-9290-1e3f7a16f28e/Alignment_Mapping.png)

This mapping function is learned by the model, whether that is a transformer, RNN, or other model. If we denote the mapping function as A, A(j) = i means that yi is aligned with xi.

Consider this simple English to French translation example:

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/9bd8e84b-b103-46e1-8419-ad1536252de1/image.png)

A simple mapping function that makes logical sense in translation is A(1) = 1, A(2) = 2, and A(3) = 3.

The French translation of “I” is “Je,” the translation of “am” is “suis,” and the translation of “happy” is in fact “content.”

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/73c4b381-c4c7-4843-8b82-17e4eaf72f56/image.png)

Here, it is important to remember one input token can be aligned with more than one target token, and vice versa.

---

### Business as usual? No thanks.

[![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/4ddd20c1-e262-4990-a7c3-25bdfc97887f/Beehiiv_September_2025_3.png)](https://www.morningbrew.com/subscribe?utm_campaign=XIAEVJHTGJ&utm_medium=paid_newsletter&utm_source=beehiiv&_bhiiv=opp_b44dad3d-5334-4c19-b4cb-ae59366ae343_fbd824b6)

The problem with most business news? It’s too long, too boring, and way too complicated.

[Morning Brew](https://www.morningbrew.com/subscribe?utm_campaign=XIAEVJHTGJ&utm_medium=paid_newsletter&utm_source=beehiiv&_bhiiv=opp_b44dad3d-5334-4c19-b4cb-ae59366ae343_fbd824b6) fixes all three. In five minutes or less, you’ll catch up on the business, finance, and tech stories that actually matter—written with clarity and just enough humor to keep things interesting.

It’s quick. It’s free. And it’s how over 4 million professionals start their day. Signing up takes less than 15 seconds—and if you’d rather stick with dense, jargon-packed business news, you can always unsubscribe.

[Check it out](https://www.morningbrew.com/subscribe?utm_campaign=XIAEVJHTGJ&utm_medium=paid_newsletter&utm_source=beehiiv&_bhiiv=opp_b44dad3d-5334-4c19-b4cb-ae59366ae343_fbd824b6)

---

### Alignment’s Application in AI

Alignment is most commonly required in sequence-to-sequence tasks, where one sequence (like a sentence in one language) must be mapped to another (like its translation in another language).

- In machine translation, alignment makes sure that the translated words are placed correctly according to their meaning and grammatical requirements.
- In speech recognition, alignment helps map audio frames to words.
- In image captioning, alignment helps connect visual features to the correct words from a caption.

Without alignment the outputs might be correct but semantically inaccurate, losing their intended meaning.

#### Alignment in Early Models

Before Transformers, Recurrent Neural Networks (RNNs) and Encoder-Decoder Networks such as Autoencoders were used for translation.

These networks faced major difficulties with alignment due to their architecture.

The encoder compresses the entire input into a single vector, and the decoder generates output step by step using only this vector.

But, for example, when the input is a long sentence, the model finds it hard to decide which source word should align with which target word.

To fix this, the concept of “attention” was developed, which allowed the models to look back at the encoder states while generating outputs. But as mentioned before, this needed to be solved as a separate task and was not a natural part of the model itself.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/41f48f10-5430-4eee-939d-6e0ba84cf4ab/Autoencoder__1_.png)

#### Alignment in Transformers: Self-Attention

With transformers, the alignment problem is handled in an elegant way. “Attention” was built into the architecture itself.

In simple terms, transformers turn words into vectors, process them through attention blocks and neural networks, and output the most likely next word.

When you type a sentence, it goes into the input layer of the transformer, where the textual prompt undergoes tokenization (broken into small pieces of words) and is combined with positional encodings.

These modified vectors are called positional embeddings. These positional embeddings are stacked together to form a matrix:

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/fd6f67e2-46cc-4e20-9eae-d2dd39ca6d74/image.png)

The embeddings then pass through several transformer blocks.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/60c02b87-1a5b-429f-a955-c0a92086a25b/Transformer_Input_Example.png)

Each transformer block has two main parts:

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/e51a3e67-463c-4e88-b332-45b706a16a65/Copy_of_Transformer_Block__2_.png)

The first part of the block is a special type of attention mechanism called Multi-Headed Attention (MHA) and layer normalization (LN) with residual connections. This is where the model decides which word should “pay attention” to which word.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/84e03f81-d285-40d4-a1f9-1011d367d504/image.png)

The second part is the Feed-Forward Neural Network (FFN), where a small neural network helps the model learn the information further, building richer representations. A full equation for the block is shown below.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/f3107c68-2c74-4d7c-9739-90e24bc706e3/image.png)

After passing through all the blocks, the output is passed to a softmax function, which computes a score for each word called a “Logit.”

The prediction for the next token is then determined by selecting the maximal logit value—that is, the highest-scoring word is chosen as the next output.

---

### [Level Up Your Career With AI, But Simple PRO](https://www.aibutsimple.com/upgrade?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=alignment-in-transformers-simply-explained)

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/281697a8-c915-435c-a68b-43374b671698/aibutsimple_academy.png)

Supporting us by purchasing a PRO membership will get you:

- Exclusive Hands-on Tutorials With Code
- Full Math Explanations and Beautiful Visualizations
- Ad-Free Reading for an Uninterrupted Learning Experience

[Break into AI or sharpen your edge by joining today!](https://www.aibutsimple.com/upgrade?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=alignment-in-transformers-simply-explained)

---

### A Mathematical View of Alignment

In 2024, a paper led by Aubry and his team, titled “[Transformer Alignment in Large Language Models](https://arxiv.org/html/2407.07810v1?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=alignment-in-transformers-simply-explained),” discuss new findings of alignment math properties.

The recent work describes alignment using linearization of transformer blocks. The key idea is that the embeddings evolve across layers in a structured way, and that alignment can be studied by examining how these transformations act on small changes.

One of the “structured ways” in which the embeddings evolve across transformer layers is shown by linearity (i.e., a straight line path, predictable) or exponentially distanced trajectories (i.e., change between layers grows exponentially) of intermediate layers when going from layer to layer.

#### Transformer Block as a Residual Map

As described earlier, a transformer block with residual connections can be simply written as:

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/f81c5614-b083-46cf-91d4-7308cc95b250/image.png)

Where X(t) is the sequence of embeddings, and Ft is the nonlinear residual mapping (attention + FFN).

This highlights the view of the transformer as a discrete dynamical system for which residual Jacobians can be calculated.

#### Residual Jacobians

For a token embedding x, the [Residual Jacobian](https://en.wikipedia.org/wiki/Jacobian_matrix_and_determinant?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=alignment-in-transformers-simply-explained) (Jt) is given by:

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/21fd3b2c-b06f-43cc-ab87-ae29f7f87577/image.png)

This captures how small perturbations to the embedding are transformed by the residual block.

#### Misalignment

Using [Singular Value Decomposition](https://en.wikipedia.org/wiki/Singular_value_decomposition?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=alignment-in-transformers-simply-explained)[(SVD)](https://en.wikipedia.org/wiki/Singular_value_decomposition?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=alignment-in-transformers-simply-explained):

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/07d740ed-3e08-4f16-9848-5a335a8c4599/image.png)

Above, Ut and Vt are components of the SVD of the Jacobian.

The alignment across layers can be studied by comparing the leading singular vector and the misalignment between two layers. Between layers i and j, misalignment is given by:

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/2fa9a0b0-3e2c-48db-84e4-efcb901fc4fb/image.png)

Where Mij compares the top-k singular vectors and ||…||F is the Frobenius norm.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/94a6b626-71d3-405e-a1a3-22f1469f4376/Alignment_Vectors.png)

A low misalignment means the main directions of transformation remain consistent across layers, a property correlated with strong model performance.

To check if embeddings evolve in a straight line across layers, a [Line-Shape Score (LSS)](https://arxiv.org/pdf/2102.09235?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=alignment-in-transformers-simply-explained) is computed:

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/4444df65-5395-4025-9d88-5b1b38822338/image.png)

Where x(t) = x(t+1) - x(t). A smaller LSS means trajectories are nearly linear, which is another important property of transformer alignment.

### Why is Attention Important?

Attention plays an important role in alignment. If the attention focuses on the wrong words, the outputs can become misleading or even completely lose their meaning.

Therefore, efforts are taken to improve alignment in transformers by guiding the attention mechanisms to pay more weight to the relevant parts of the input.

### Conclusion

Alignment is a fundamental challenge in sequence-to-sequence tasks. Transformers solve this using the attention mechanism, which helps them automatically learn the alignments between inputs and outputs.

Mathematically, alignment is evident from the consistent singular directions across layers, exponential growth patterns in embeddings, and linear trajectories. These are not just mathematical properties, but they also strongly correlate with model quality and performance!

---

Here’s a special thanks to our [biggest supporters](https://www.aibutsimple.com/upgrade?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=alignment-in-transformers-simply-explained):

- Sushant Waidande
- Sai Krishna Pashapu

---

If you enjoy our content, consider supporting us so we can keep doing what we do. Please share this with a friend!

Want to reach 50000+ ML engineers? Let’s work together:

[WORK WITH US 📈](https://www.passionfroot.me/aibutsimple?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=alignment-in-transformers-simply-explained)

Feedback, inquiries? Send us an email at [edwin@aibutsimple.com](mailto:edwin@aibutsimple.com).

If you like, you can also donate to our team to push out better newsletters every week!

[Support us ☕](https://ko-fi.com/aibutsimple?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=alignment-in-transformers-simply-explained)

---

That’s it for this week’s issue of AI, but simple. See you next week!

—AI, but simple team