# Dimensionality Reduction: t-SNE and UMAP

#81 Dec 15, 2025 Theory

https://www.aibutsimple.com/p/the-chain-rule-is-the-backbone-of-backpropagation-it-allows-us-to-calculate-derivatives-of-nested-fu

Modern machine learning models operate in extremely high-dimensional spaces.

A single image can contain tens of thousands of pixel values, word embeddings often have hundreds of dimensions, and the hidden layers of deep networks may contain thousands of learned features.

While these high-dimensional representations are powerful, they are also very difficult to interpret, visualize, and analyze directly.

Dimensionality reduction is the process of transforming high-dimensional data into a lower-dimensional space while preserving the most important structure in the data.

By mapping data into two or three dimensions, we can visually explore clusters, analyze class separability, detect outliers, and understand how machine learning models organize information internally.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/ef48d74d-dd70-442c-a5d8-78e03d151808/High_Dimensional_Space_Dimensionality_Reduction__2_.png)

While [Principal Component Analysis (PCA)](https://www.aibutsimple.com/p/principle-component-analysis-pca-simply-explained?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=dimensionality-reduction-t-sne-and-umap) provides a linear approach to dimensionality reduction, many real-world datasets lie on nonlinear spaces.

To address this, two powerful nonlinear techniques, t-SNE (t-Distributed Stochastic Neighbor Embedding) and UMAP (Uniform Manifold Approximation and Projection), have become standard tools for modern data visualization.

Before we continue, let’s unpack some general dimensionality reduction terms we should have an understanding of:

- Manifolds, knowing that they are complex spaces that locally look like regular Euclidean space. For instance, Earth’s surface is curved globally but is flat enough locally for maps. The key assumption is that real-world data is concentrated near a low-dimensional manifold.
- Neighborhoods, a way in which data points are considered “close” or “similar” to each other. This can be within a radius, a hyperparameter k, or even probability-based, like in t-SNE.
- Perplexity, a t-SNE parameter controlling how neighborhoods are defined. Low perplexity gives local neighborhoods, while high perplexity gives larger, more global neighborhoods.
- [KL divergence (Kullback–Leibler divergence)](https://www.aibutsimple.com/p/kl-divergence-simply-explained?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=dimensionality-reduction-t-sne-and-umap), a way to measure how different two probability distributions are. You can read more about KL divergence in our previous issue.

---

### The Future of Shopping? AI + Actual Humans.

[![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/32cda317-7f88-43d7-8db1-5b6ffdf33f5c/Affiliate_3.0_Gif-Levanta__2_.gif)](https://get.levanta.io/ai?utm_source=beehiiv&utm_medium=paidnewsletter&utm_campaign=XIAEVJHTGJ&utm_term=prospecting&utm_content=affiliate_ai_report_g1&_bhiiv=opp_a10d8f9d-fafd-483e-b0ea-f58d7b2c9c9c_9dcd0883)

AI has changed how consumers shop, but people still drive decisions. [Levanta’s](https://get.levanta.io/ai?utm_source=beehiiv&utm_medium=paidnewsletter&utm_campaign=XIAEVJHTGJ&utm_term=prospecting&utm_content=affiliate_ai_report_g1&_bhiiv=opp_a10d8f9d-fafd-483e-b0ea-f58d7b2c9c9c_9dcd0883) research shows affiliate and creator content continues to influence conversions, plus it now shapes the product recommendations AI delivers. Affiliate marketing isn’t being replaced by AI, it’s being amplified.

[Download the full report](https://get.levanta.io/ai?utm_source=beehiiv&utm_medium=paidnewsletter&utm_campaign=XIAEVJHTGJ&utm_term=prospecting&utm_content=affiliate_ai_report_g1&_bhiiv=opp_a10d8f9d-fafd-483e-b0ea-f58d7b2c9c9c_9dcd0883)

---

### Applications in Machine Learning

In today’s machine learning systems, t-SNE and UMAP are used primarily as visualization and exploratory analysis tools rather than as preprocessing steps for training.

Some common applications include:

- Visualizing neural network embeddings, like CNN feature maps for images, Transformer embeddings for text, and latent spaces of autoencoders. The embeddings are routinely projected into 2D using t-SNE or UMAP.
- Cluster discovery and data exploration, including customer segmentation, biological data analysis, and anomaly detection.
- Model debugging and interpretability: Researchers use t-SNE and UMAP to inspect how well a model separates classes in its hidden layers and to detect representation collapse.
- Scientific research, like single-cell RNA sequencing, brain activity analysis, and physics simulations, often produces extremely high-dimensional measurements that must be visualized effectively.

In practice, if PCA fails to reveal clear structure, t-SNE and UMAP are usually the next tools of choice.

### Foundation: Manifolds and Neighborhoods

Both t-SNE and UMAP are built on a shared mathematical assumption:

High-dimensional data lies on a lower-dimensional nonlinear manifold embedded in the original space.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/de8646b7-9cfe-4077-a48f-50112ee51eef/3D_Space_with_2D_Manifold_Dimensionality_Reduction.png)

Rather than preserving global distances, these algorithms prioritize local neighborhood relationships. If two data points are close in high-dimensional space, they should remain close in the low-dimensional embedding.

This local-preservation principle is the key idea that differentiates t-SNE and UMAP from PCA.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/febfa975-e265-469f-be14-0c7612fe004b/PCA_Linear_vs_t-SNEUMAP_Nonlinear_Dimensionality_Reduction.png)

### t-SNE: t-Distributed Stochastic Neighbor Embedding

t-SNE converts distances between data points into pairwise probabilities, both in the original space and in the low-dimensional space.

It then finds the low-dimensional embedding that minimizes the mismatch (KL divergence) between these two probability distributions.

In simple terms: t-SNE tries to arrange points in 2D such that neighbors in high dimensions remain neighbors in low dimensions.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/e6e57047-7049-4ee1-81fa-50a87b9a230c/High_Dimensional_vs_Low_Dimensional_Distributions_Dimensionality_Reduction.png)

### High-Dimensional Probability Distribution

Given a dataset (x1, x2, ..., xn), t-SNE defines a conditional probability:

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/71d09702-4683-4e43-96e0-0c53502d1c78/image.png)

This represents the probability that (xj) would be chosen as a neighbor of (xi). A symmetric joint probability is then formed:

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/4fa6d4d0-d8a9-4ac3-af18-c5916c8cf856/image.png)

The bandwidths i are chosen automatically using the perplexity, which controls the effective neighborhood size.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/975fbe75-7a29-467f-a4f4-29b9d8cabd71/High_vs_Low_Perplexity_Dimensionality_Reduction.png)

### Low-Dimensional Probability Distribution

Let (y1, y2, ..., yn) be the 2D embeddings. t-SNE defines similarities using a [Student’s t-distribution](https://en.wikipedia.org/wiki/Student%27s_t-distribution?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=dimensionality-reduction-t-sne-and-umap):

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/4a704c46-99e7-4111-b3e8-2d6b6009b5d8/image.png)

The heavy tails of the t-distribution solve the crowding problem, allowing distant points to be pushed far apart.

### Optimization Using KL Divergence

t-SNE minimizes the Kullback–Leibler divergence between the two distributions:

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/277952ad-134d-432a-8f5f-c4cda39e3848/image.png)

This ensures that point pairs that are neighbors in high dimensions remain neighbors in the low-dimensional visualization. Optimization is performed using gradient descent.

### Strengths and Limitations of t-SNE

Some strengths of t-SNE include:

- Excellent local structure preservation
- Produces clear and visually separated clusters
- Very powerful for small and medium-sized datasets
- Widely used in NLP and vision research

However, limitations of t-SNE include:

- Computationally expensive for large datasets
- Global distances between clusters are not meaningful
- Layout changes across runs
- Cannot easily embed new points
- Sensitive to perplexity and learning rate

### UMAP: Uniform Manifold Approximation and Projection

UMAP is based on ideas from manifold learning and algebraic topology. Instead of building full probability distributions over all point pairs (as t-SNE does), UMAP constructs a sparse graph representation of local neighborhoods and preserves this structure in low dimensions.

UMAP assumes:

- The data lies on a smooth manifold
- Local neighborhood relationships define the global structure

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/0ec912bb-b59f-488d-a276-7b802a8b09fb/Before_and_After_t-SNE_or_UMAP_dimensionality_reduction.png)

#### Step 1: k-Nearest Neighbor Graph

For each data point, UMAP identifies its k-nearest neighbors. A weighted graph is constructed where edges represent fuzzy membership strengths instead of binary connectivity.

#### Step 2: Fuzzy Simplicial Complex

The local graphs are combined into a global fuzzy topological structure that approximates the underlying manifold geometry of the data.

#### Step 3: Low-Dimensional Optimization

UMAP finds a low-dimensional embedding that minimizes the cross-entropy between the high-dimensional and low-dimensional fuzzy graphs:

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/a860811a-55d2-4a1c-9229-80ac6b8164a2/image.png)

This encourages connected points to stay close and disconnected points to separate.

### Strengths and Limitations of UMAP

Some strengths of UMAP include:

- It is much faster than t-SNE
- Preserves both local and partial global structure
- Stable and reproducible
- Scales to millions of samples
- Supports embedding of new unseen data
- Suitable for supervised and semi-supervised learning

Some limitations of UMAP include:

- Sensitive to hyperparameters (n_neighbors, min_dist)
- Distances are not strictly metric
- May introduce artificial clusters if misused
- Still nonlinear and non-invertible

---

### [Level Up Your Career With AI, But Simple PRO](https://www.aibutsimple.com/upgrade?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=dimensionality-reduction-t-sne-and-umap)

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/281697a8-c915-435c-a68b-43374b671698/aibutsimple_academy.png)

Supporting us by purchasing a PRO membership will get you:

- Exclusive Hands-on Tutorials With Code
- Full Math Explanations and Beautiful Visualizations
- Ad-Free Reading for an Uninterrupted Learning Experience

[Break into AI or sharpen your edge by joining today!](https://www.aibutsimple.com/upgrade?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=dimensionality-reduction-t-sne-and-umap)

---

### t-SNE vs UMAP: Conceptual Comparison

| Aspect | t-SNE | UMAP |
| Mathematical basis | Probability & KL divergence | Topology & manifolds |
| Speed | Slow | Very fast |
| Local structure | Excellent | Excellent |
| Global structure | Weak | Moderate |
| Stability | Low | High |
| Large datasets | Not suitable | Highly suitable |
| New data embedding | No | Yes |

### Example: Visualizing Word Embeddings

Suppose we train word embeddings on a corpus. Each word is represented by a 300-dimensional vector. By applying:

- t-SNE, we can often observe tight semantic clusters (fruits, vehicles, professions, etc), but cluster spacing is unreliable.
- UMAP, we can observe similar clusters but also meaningful global relationships between groups.

Both methods reveal information that is completely invisible in the original high-dimensional space.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/c6123e35-cc63-41a7-8403-25b078d1b619/T-SNE_vs_UMAP_dimensionality_reduction.png)

### Interpretation Pitfalls

Although t-SNE and UMAP produce visually appealing and highly informative visualizations, their outputs must be interpreted with caution.

One common misconception is to treat the distances between clusters in the low-dimensional embedding as meaningful measures of similarity.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/4ccf1b15-29e5-4b89-83bb-30cb5d8106a0/image.png)

In reality, inter-cluster distances are not guaranteed to correspond to true distances in the original high-dimensional space, especially for t-SNE, which prioritizes local neighborhood preservation over global structure.

Another important consideration is that cluster sizes in t-SNE or UMAP plots are not statistically meaningful.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/2dc9721a-ceab-4d6c-9ecc-0a110d7ad179/image.png)

The apparent size or density of a cluster in the visualization does not necessarily reflect the number of data points, variance within the cluster, or importance of that group in the original dataset.

These visual properties are artifacts of the optimization process rather than direct representations of data statistics.

Additionally, both t-SNE and UMAP rely on stochastic optimization and random initialization.

As a result, different runs, particularly with t-SNE, can lead to noticeably different layouts, even when applied to the same dataset.

While local neighborhoods are often preserved, the global arrangement of clusters may rotate, flip, or shift between runs. To avoid this, a fixed random seed can be used while using the method.

For these reasons, t-SNE and UMAP are mostly used as exploratory tools for visualization. They are great for gaining intuition, generating hypotheses, and inspecting representations, but they cannot be used as evidence or proof of structure without additional quantitative analysis.

### When to Use Which Method

Principal Component Analysis (PCA) is most appropriate to use when simplicity, interpretability, and mathematical transparency are important.

Because PCA is a linear method, each principal component has a clear meaning in terms of variance explained, making it well suited for datasets that are approximately linear or when understanding the contribution of original features is considered important.

t-SNE is best suited for situations where the primary goal is high-quality visualization of local structure.

It excels at revealing tightly separated clusters and uncovering fine-grained neighborhood relationships, particularly in small to medium-sized datasets. However, it is less suitable when reproducibility, scalability, or embedding new data points is required.

UMAP should be preferred when working with very large datasets or when both speed and stability are important.

Its ability to preserve local structure while maintaining some global organization, combined with its scalability and support for out-of-sample embeddings, makes UMAP a strong default choice for modern machine learning workflows and large-scale exploratory analysis.

### Conclusion

t-SNE and UMAP have transformed the way researchers visualize and interpret high-dimensional data.

By shifting the goal from preserving variance to preserving neighborhood relationships, these methods uncover structures that linear techniques cannot easily capture.

t-SNE remains the gold standard for high-quality small-scale visualizations, while UMAP has become the default choice for large-scale exploratory analysis.

Together, they form essential tools in modern machine learning for easy human visual understanding and interpretation.

---

Here’s a special thanks to our [biggest supporters](https://www.aibutsimple.com/upgrade?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=dimensionality-reduction-t-sne-and-umap):

- Sushant Waidande
- Sai Krishna Pashapu
- Bob Boncella

---

If you enjoy our content, consider supporting us so we can keep doing what we do. Please share this with a friend!

Want to reach 50000+ ML engineers? Let’s work together:

[WORK WITH US 📈](https://www.passionfroot.me/aibutsimple?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=dimensionality-reduction-t-sne-and-umap)

Feedback, inquiries? Send us an email at [edwin@aibutsimple.com](mailto:edwin@aibutsimple.com).

If you like, you can also donate to help our team push out better newsletters every week!

[Support us ☕](https://ko-fi.com/aibutsimple?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=dimensionality-reduction-t-sne-and-umap)

---

That’s it for this week’s issue of AI, but simple. See you next week!

—AI, but simple team