Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

PII Removal › Visualizations

Michael BrenndoerferJanuary 10, 20266 min read

Language AI HandbookAll chapters

Part 20 of 67Data Curation

Chapter 146PII Removal › Visualizations

Reading section 7 of 10 · Visualizations · 6 minSection 7 of 10 · 6 min

Section 7 of 10 · 6 min read

PII Removal › VisualizationsOne section · Sections in this chapter7 / 10 ↓

Sections in this chapter10

Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

The following plots illustrate key aspects of PII detection: the precision and recall trade-off per category, the distribution of PII types and densities in a typical web corpus, and the recall gains from adding NER to a pattern-only pipeline.

Out[17]:

Visualization

Grouped bar chart comparing precision and recall for EMAIL, PHONE, and SSN detection categories. — Precision and recall for EMAIL, PHONE, and SSN detection by the pattern-based pipeline, computed on a 155-example synthetic test set with known ground truth. All three categories score 1.00 on this controlled sample. The perfect result is a pipeline sanity check rather than evidence of production accuracy, because real corpora contain ambiguous digit sequences and more varied formats.

Out[18]:

Visualization

Bar chart showing count of documents containing each PII category type in a simulated corpus. — Distribution of PII categories across a simulated 10,000-document web corpus. Email addresses are the most frequent PII category, appearing in roughly 3% of documents. SSNs and credit card numbers are rare, appearing in less than 0.5% of documents. The majority of documents (91%) contain no detected PII, showing that PII removal affects a small but critical fraction of the corpus.

Histogram of per-document PII token density with a vertical line marking the 10% removal threshold. — PII token density distribution across documents in a simulated corpus. Most documents cluster near zero density, representing clean text. A small tail of PII-dense documents extends to higher densities. The dashed red line at 10% marks a common threshold for document-level removal, showing that only a small fraction of documents would be removed by this policy.

Out[19]:

Visualization

Grouped bar chart comparing recall of pattern-only versus hybrid pipeline across four document types. — Recall comparison between a pattern-only pipeline and a hybrid pipeline (pattern plus NER) across four document categories. Adding NER provides the largest gains on forum posts and social media, where names appear in informal contexts without standard formatting. Structured documents (forms and templates with predefined fields) show a smaller improvement because their PII is predominantly format-structured and well-captured by regex. The recall gap between pipeline types is largest where free-form name mentions are most prevalent.

Out[20]:

Visualization

Line chart showing fraction of documents retained as the PII density removal threshold increases from 0 to 30 percent. — Corpus retention rate as a function of the document removal threshold, showing the trade-off between privacy protection and data preservation. At a 5% threshold, roughly 4-6% of documents are removed. At a 15% threshold, only the most egregiously PII-dense documents are removed. The curve flattens above 15% because very few documents have PII densities that high. Practitioners typically choose a threshold in the 8-12% range to balance coverage with minimal data loss.