Regular Expressions › Summary

Regular expressions provide a compact, powerful language for pattern matching in text. They occupy a specific and important niche in the NLP practitioner's toolkit: they excel at extracting and transforming text with rigid structures, such as emails and phone numbers, dates and URLs, or log entries. They are not the right tool for problems involving nesting, context-dependence, or semantic understanding. Used appropriately, they are elegant and fast. Used inappropriately, they become unmaintainable liabilities.

The mental model that will serve you best is to think of regex patterns as specifications for text structure. You are describing the shape of what you are looking for, not writing a procedure to find it. The regex engine handles the search procedure. Your job is to describe the structure precisely enough that only the things you want match, and everything else does not. This requires careful attention to anchors (to prevent partial matches), character classes (to be specific about what is acceptable), quantifiers (to specify repetition correctly), and lookarounds (to use context as a filter without including it in results).

You've learned:

• Metacharacters: . matches any character, [] defines character classes, ^ and $ anchor to positions
• Quantifiers: *, +, ?, {n,m} control repetition; add ? for lazy matching
• Groups: () captures text, (?:) groups without capturing, (?P<name>) names captures
• Lookarounds: (?=), (?!), (?<=), (?<!) match positions based on context
• Backreferences: \1, \2 refer back to captured groups
• Flags: re.IGNORECASE, re.MULTILINE, re.DOTALL, re.VERBOSE modify behavior

Key practical patterns for NLP:

• Emails: \b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b
• URLs: https?://\S+
• Hashtags/Mentions: #\w+, @\w+
• Word boundaries: \bword\b for whole-word matching

Best practices:

• Always use raw strings: r"pattern"
• Compile patterns used repeatedly
• Avoid nested quantifiers that cause catastrophic backtracking
• Use string methods for simple operations
• Use proper parsers for structured formats like HTML or JSON
• Use re.VERBOSE with comments for any non-trivial pattern
• Use named groups in patterns you will maintain over time
• Test patterns on adversarial inputs and the happy path

In the next chapter, we'll explore sentence segmentation, where regex plays a supporting role in identifying sentence boundaries. You'll see how the word boundary anchor \b and end-of-sentence punctuation patterns combine with heuristics to split continuous text into discrete sentences, a foundational step in almost every NLP pipeline.