t-SNE (t-Distributed Stochastic Neighbor Embedding) › Visualizing t-SNE

Michael BrenndoerferJuly 30, 20256 min read
Machine Learning from ScratchAll chapters
Part 6 of 9Explainability
Chapter 49t-SNE (t-Distributed Stochastic Neighbor Embedding) › Visualizing t-SNE
Reading section 3 of 8 · Visualizing t-SNE · 6 minSection 3 of 8 · 6 min

Book contents

Machine Learning from Scratch9 parts · 66 chaptersView the whole book →
Active chapter 49 of 66

Section 3 of 8 · 6 min read

t-SNE (t-Distributed Stochastic Neighbor Embedding) › Visualizing t-SNEOne section · Sections in this chapter3 / 8 ↓
Sections in this chapter8
Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

Let's create visualizations to understand how t-SNE works with different types of data structures. We'll start with a simple example using synthetic data to demonstrate the key concepts.

In[3]:
Code
import sys
from pathlib import Path

import numpy as np
from matplotlib.colors import BoundaryNorm, ListedColormap

for parent in [Path.cwd(), *Path.cwd().parents]:
    helper_dir = (
        parent / "apps/web/content/notebooks/1_machine-learning-from-scratch"
    )
    if (helper_dir / "mls_plot_style.py").exists():
        if str(helper_dir) not in sys.path:
            sys.path.insert(0, str(helper_dir))
        break
else:
    raise FileNotFoundError("Could not locate mls_plot_style.py")

from mls_plot_style import PALETTE, polish_axes, use_mls_style

# Set random seed for reproducibility
np.random.seed(42)

binary_cmap = ListedColormap([PALETTE["primary"], PALETTE["amber"]])
digit_cmap = ListedColormap(
    [
        PALETTE["primary"],
        PALETTE["amber"],
        PALETTE["teal"],
        PALETTE["rose"],
        PALETTE["purple"],
        PALETTE["coral"],
        PALETTE["primary_mid"],
        PALETTE["amber_soft"],
        PALETTE["teal_soft"],
        PALETTE["rose_soft"],
    ]
)
digit_norm = BoundaryNorm(np.arange(-0.5, 10.5, 1), digit_cmap.N)
Out[4]:
Visualization
Scatter plot of 300 points in three original linear clusters.
Original 2D data with three linear clusters.
Scatter plot of 300 points in two original concentric circles.
Original 2D data with a circular structure.
Scatter plot of 300 points in two original moon-shaped clusters.
Original 2D data with moon-shaped clusters.
Out[5]:
Visualization
Scatter plot of the t-SNE embedding of the linear-cluster data.
t-SNE embedding of the linear clusters.
Scatter plot of the t-SNE embedding of the circular data.
t-SNE embedding of the circular structure.
Scatter plot of the t-SNE embedding of the moon-shaped data.
t-SNE embedding of the moon-shaped clusters.

Now let's examine how different perplexity values affect the t-SNE visualization:

Out[6]:
Visualization
Scatter plot of the moon-shaped t-SNE embedding with perplexity 5.
Moon-shaped data embedded with perplexity 5, emphasizing very local neighborhoods.
Scatter plot of the moon-shaped t-SNE embedding with perplexity 15.
Moon-shaped data embedded with perplexity 15, emphasizing local neighborhoods.
Scatter plot of the moon-shaped t-SNE embedding with perplexity 30.
Moon-shaped data embedded with perplexity 30, balancing local and broader structure.
Out[7]:
Visualization
Scatter plot of the moon-shaped t-SNE embedding with perplexity 50.
Moon-shaped data embedded with perplexity 50, showing broader neighborhood structure.
Scatter plot of the moon-shaped t-SNE embedding with perplexity 100.
Moon-shaped data embedded with perplexity 100, emphasizing global patterns.
Scatter plot of the moon-shaped t-SNE embedding with perplexity 200.
Moon-shaped data embedded with perplexity 200, emphasizing the broadest patterns.

Let's also visualize the effect of t-SNE on high-dimensional data by projecting it back to 2D:

Out[8]:
Visualization
Scatter plot showing t-SNE projection of 20-dimensional data with clear class separation.
t-SNE projection of 20-dimensional synthetic data showing clear class separation through non-linear dimensionality reduction. The algorithm successfully reveals the underlying cluster structure that exists in the high-dimensional space.
Scatter plot showing PCA projection of 20-dimensional data with less clear class separation.
PCA projection of the same 20-dimensional data showing linear dimensionality reduction. While PCA captures the maximum variance, it fails to reveal the clear class separation that t-SNE successfully identifies.