MinHash › Visualizations

Michael BrenndoerferJanuary 10, 20266 min read
Language AI HandbookAll chapters
Part 20 of 67Data Curation
Chapter 143MinHash › Visualizations
Reading section 8 of 13 · Visualizations · 6 minSection 8 of 13 · 6 min

Book contents

Language AI Handbook67 parts · 513 chaptersView the whole book →
Active chapter 143 of 513

Section 8 of 13 · 6 min read

MinHash › VisualizationsOne section · Sections in this chapter8 / 13 ↓
Sections in this chapter13
Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

Let's visualize two key properties of MinHash: how the similarity estimate accuracy improves with more hash functions, and the S-curve behavior of LSH banding.

Out[14]:
Visualization
Line plot showing decreasing MinHash estimation error as number of hash functions increases from 8 to 512 for four Jaccard values.
Mean absolute error of MinHash Jaccard similarity estimates as a function of the number of hash functions, for four different true Jaccard values. Error decreases with more hash functions following the theoretical 1/sqrt(k) rate. At 128 hash functions the mean error falls below 0.04 for all similarity levels, making MinHash accurate enough for practical deduplication decisions.
Out[15]:
Visualization
S-curve plot of LSH detection probability versus Jaccard similarity for three band configurations.
LSH candidate detection probability as a function of Jaccard similarity for three band configurations with different total hash budgets. Each S-shaped curve transitions sharply around a threshold determined by the band parameters, with vertical dotted lines marking each approximate threshold. Higher r (rows per band) produces steeper transitions; higher b (number of bands) shifts the threshold lower.
S-curve plot comparing four band or row splits for 128 total hash functions.
LSH probability curves for a fixed 128-hash budget, split into four different band configurations. Fewer bands with more rows per band (b=8, r=16) produce a steep S-curve at a high threshold, while many bands with fewer rows (b=64, r=2) detect lower-similarity pairs with a shallower transition. This tradeoff lets practitioners tune detection sensitivity.
Out[16]:
Visualization
Heatmap matrix of MinHash pairwise similarity scores for 10 documents, revealing two clusters of near-duplicates.
MinHash pairwise similarity estimates for a 10-document corpus containing two groups of near-duplicates. Documents 1 through 5 are variations of a base sentence about machine learning; documents 6 through 10 are variations of a base sentence about neural networks. High estimated similarity (green) appears within each group, while cross-group pairs show near-zero similarity (red), showing how MinHash LSH exploits these clusters to find near-duplicates efficiently.

The heatmap reveals the two clusters of near-duplicates clearly. Documents 1 through 5 are paraphrases of the first base text; documents 6 through 10 are paraphrases of the second. Within each cluster, MinHash estimates high similarity (green). Across clusters, similarity is near zero (red). This block structure is exactly what MinHash LSH exploits: the LSH bands quickly identify the within-cluster pairs as candidates while ignoring the cross-cluster pairs.