Regular Expressions › Substitution and Transformation
Language AI HandbookAll chapters Regular Expressions › Substitution and TransformationOne section · Sections in this chapter12 / 21 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
The re.sub() function replaces matches with new text. You can use backreferences in the replacement string. Substitution is one of the most important regex operations for NLP preprocessing because cleaning text nearly always means replacing certain patterns with something else. You might replace URLs with a <URL> token so the model knows a URL was present without exposing the actual domain. You might replace all numbers with <NUM> to reduce vocabulary size. You might normalize whitespace, remove punctuation, or redact personally identifiable information (PII) before storing documents in a database.
The real power of re.sub() comes from its two modes: static replacement strings and callable replacement functions. A static string can reference captured groups using \1, \2, or named groups with \g<name>. A callable receives the match object and can execute arbitrary Python logic to compute the replacement, enabling transformations that would be impossible with a pure string replacement.
text = "Contact john.doe@email.com or jane.smith@company.org"
# Simple replacement
redacted = re.sub(r"\S+@\S+", "[EMAIL REDACTED]", text)
# Using backreferences in replacement
# Swap first and last name in email
swapped = re.sub(r"(\w+)\.(\w+)@", r"\2.\1@", text)
# Using a function for complex replacements
def mask_email(match):
email = match.group()
name, domain = email.split("@")
return f"{name[0]}***@{domain}"
masked = re.sub(r"\S+@\S+", mask_email, text)Original: 'Contact john.doe@email.com or jane.smith@company.org' Simple replacement: Contact [EMAIL REDACTED] or [EMAIL REDACTED] Backreference swap (\1.\2@ → \2.\1@): Contact doe.john@email.com or smith.jane@company.org Function-based masking: Contact j***@email.com or j***@company.org