K-means Clustering › Visualizing K-means Clustering

Michael BrenndoerferAugust 2, 20259 min read
Machine Learning from ScratchAll chapters
Part 7 of 9Unsupervised Learning
Chapter 53K-means Clustering › Visualizing K-means Clustering
Reading section 3 of 15 · Visualizing K-means Clustering · 9 minSection 3 of 15 · 9 min

Book contents

Machine Learning from Scratch9 parts · 66 chaptersView the whole book →
Active chapter 53 of 66

Section 3 of 15 · 9 min read

K-means Clustering › Visualizing K-means ClusteringOne section · Sections in this chapter3 / 15 ↓
Sections in this chapter15

Let's visualize how K-means clustering works step by step. We'll generate synthetic data with three distinct clusters and show the iterative process of centroid updates and cluster assignments.

Out[5]:
Visualization
Scatter plot showing initial random centroid placement with unassigned data points.
Step 1: Random initialization of cluster centroids. The algorithm starts with three randomly placed centroids (marked with ''x'') in the feature space. At this stage, no cluster assignments have been made yet, and all data points are shown in light gray. The random placement of initial centroids can significantly impact the final clustering results, which is why K-means++ initialization is often preferred over pure random initialization.
Scatter plot showing first iteration with cluster assignments and updated centroids.
Step 2: After the first iteration, data points are assigned to their nearest centroids and centroids are updated. Each point is colored according to its assigned cluster, and the centroids have moved to better positions based on the mean of their assigned points. This demonstrates the core K-means process: assign points to nearest centroids, then update centroids to the mean of their assigned points.
Out[6]:
Visualization
Scatter plot showing third iteration with refined cluster positions.
Step 3: After three iterations, the clustering is becoming more refined. The centroids continue to move toward optimal positions, and the cluster assignments become more stable. Notice how the centroids are now positioned more centrally within their respective clusters, reducing the overall within-cluster sum of squares.
Scatter plot showing final converged clusters with decision boundaries.
Step 4: Final converged result with cluster boundaries. The algorithm has converged when centroids no longer change significantly between iterations. The black contour lines show the decision boundaries (Voronoi diagram) that separate the clusters. Each point is assigned to the cluster whose centroid is closest, creating distinct regions in the feature space.

Here you can see the iterative nature of K-means clustering in action. Watch how the algorithm starts with random centroids and progressively improves the cluster assignments and centroid positions until convergence. The final plot includes cluster boundaries to show how the algorithm partitions the feature space.

Now let's create a second visualization that shows the impact of different initialization strategies and the importance of choosing the right number of clusters:

Out[7]:
Visualization
Scatter plot of K-means clustering with k=2 and two labeled clusters.
K-means with k=2, showing under-clustering and its silhouette score.
Scatter plot of K-means clustering with k=3 and three labeled clusters.
K-means with k=3, showing under-clustering and its silhouette score.
Scatter plot of K-means clustering with k=4 and four labeled clusters.
K-means with k=4, the true cluster count, and its silhouette score.
Out[8]:
Visualization
Scatter plot of K-means clustering with k=5 and five labeled clusters.
K-means with k=5, showing the first over-clustered solution and its silhouette score.
Scatter plot of K-means clustering with k=6 and six labeled clusters.
K-means with k=6, showing additional cluster fragmentation and its silhouette score.
Scatter plot of K-means clustering with k=7 and seven labeled clusters.
K-means with k=7, showing stronger over-clustering and its silhouette score.

This second visualization shows how different values of k affect the clustering results. Notice that k=4 (the true number of clusters) achieves the highest silhouette score.