Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

t-SNE (t-Distributed Stochastic Neighbor Embedding) › Example

Michael BrenndoerferJuly 30, 20254 min read

Machine Learning from ScratchAll chapters

Part 6 of 9Explainability

Chapter 49t-SNE (t-Distributed Stochastic Neighbor Embedding) › Example

Reading section 4 of 8 · Example · 4 minSection 4 of 8 · 4 min

Section 4 of 8 · 4 min read

t-SNE (t-Distributed Stochastic Neighbor Embedding) › ExampleOne section · Sections in this chapter4 / 8 ↓

Sections in this chapter8

Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

Let's work through a concrete example using a small dataset to demonstrate how t-SNE calculations work step by step. We'll use a simple 2D dataset with 8 points divided into two clusters, showing exactly how the formulas are applied in practice.

In[9]:

Code

# Create a simple 2D dataset for demonstration
X_example = np.array(
    [[1, 1], [1, 2], [2, 1], [2, 2], [5, 5], [5, 6], [6, 5], [6, 6]]
)
y_example = np.array([0, 0, 0, 0, 1, 1, 1, 1])

Out[10]:

Console

Original 2D data:
[[1 1]
 [1 2]
 [2 1]
 [2 2]
 [5 5]
 [5 6]
 [6 5]
 [6 6]]

Labels: [0 0 0 0 1 1 1 1]

Step 1: Calculate pairwise distances

First, we compute the squared Euclidean distance $d_{ij} = ||x_i - x_j||^2$ for all pairs of points.

In[11]:

Code

# Calculate pairwise distances manually
n = X_example.shape[0]
distances = np.zeros((n, n))

for i in range(n):
    for j in range(n):
        distances[i, j] = np.sum((X_example[i] - X_example[j]) ** 2)

Out[12]:

Console

Pairwise squared distances:
[[ 0.  1.  1.  2. 32. 41. 41. 50.]
 [ 1.  0.  2.  1. 25. 32. 34. 41.]
 [ 1.  2.  0.  1. 25. 34. 32. 41.]
 [ 2.  1.  1.  0. 18. 25. 25. 32.]
 [32. 25. 25. 18.  0.  1.  1.  2.]
 [41. 32. 34. 25.  1.  0.  2.  1.]
 [41. 34. 32. 25.  1.  2.  0.  1.]
 [50. 41. 41. 32.  2.  1.  1.  0.]]

Step 2: Convert distances to probabilities

Next, we apply the Gaussian kernel formula to convert distances into conditional probabilities. For demonstration, we'll calculate $p_{j|0}$ (the probability that point 0 picks each other point as a neighbor):

p_{j|i} = \frac{\exp(-d_{ij}/2\sigma_i^2)}{\sum_{k \neq i} \exp(-d_{ik}/2\sigma_i^2)}

We use $\sigma = 1.0$ for simplicity (in practice, $\sigma$ is determined by the perplexity parameter).

In[13]:

Code

# Calculate conditional probabilities for one point (point 0)
sigma = 1.0  # Simplified - in practice, this is determined by perplexity
i = 0

# Calculate conditional probabilities p_{j|i}
p_ji = np.zeros(n)
for j in range(n):
    if i != j:
        p_ji[j] = np.exp(-distances[i, j] / (2 * sigma**2))

# Normalize
p_ji[i] = 0  # Set self-probability to 0
p_ji = p_ji / np.sum(p_ji)

Out[14]:

Console

Conditional probabilities p_{j|0} (using sigma = 1.0):
p_{0|0} = 0.0000
p_{1|0} = 0.3837
p_{2|0} = 0.3837
p_{3|0} = 0.2327
p_{4|0} = 0.0000
p_{5|0} = 0.0000
p_{6|0} = 0.0000
p_{7|0} = 0.0000

Notice how points 1, 2, and 3 (which are close to point 0 in the first cluster) have higher probabilities, while points 4-7 (in the distant second cluster) have probabilities close to zero.

Step 3: Apply complete t-SNE algorithm

Now we apply the full t-SNE algorithm, which computes all pairwise probabilities, symmetrizes them, creates a low-dimensional representation, and iteratively optimizes the positions to minimize the KL divergence.

In[15]:

Code

# Apply t-SNE to our example data
tsne_example = TSNE(n_components=2, random_state=42, perplexity=2)
X_tsne_example = tsne_example.fit_transform(X_example)

Out[16]:

Console

t-SNE result (low-dimensional coordinates):
[[ -95.73   48.14]
 [-100.53   43.68]
 [-100.19   52.94]
 [-104.99   48.48]
 [  44.72  -55.92]
 [  38.18  -55.52]
 [  45.11  -49.37]
 [  38.57  -48.98]]

Out[17]:

Visualization

Scatter plot showing original 2D data points labeled P0-P7 in two clusters. — Original 2D dataset with 8 points in two clusters. Points P0-P3 form one cluster in the lower-left region, while points P4-P7 form another cluster in the upper-right region. This is the ground truth for evaluating t-SNE's ability to preserve cluster structure.

Scatter plot showing t-SNE embedding preserving cluster separation. — t-SNE mapping result showing the algorithm preserves local structure while creating a 2D visualization. The cluster separation is maintained, with points P0-P3 remaining grouped together and points P4-P7 forming a distinct cluster.