Regular Expressions › Metacharacters: The Building Blocks
Language AI HandbookAll chapters Regular Expressions › Metacharacters: The Building BlocksOne section · Sections in this chapter4 / 21 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Regular expressions use special characters called metacharacters to represent patterns. These characters have meaning beyond their literal value. While literal characters like a or 5 match themselves, metacharacters like ., *, and [] define rules for what to match. Mastering these building blocks is the key to writing effective patterns.
The complete set of regex metacharacters is surprisingly small: . ^ $ * + ? { } [ ] \ | ( ). Every complex pattern you encounter is constructed entirely from these characters combined with literals. This is what makes regex a learnable skill rather than an arbitrary collection of incantations. Once you understand what each metacharacter does, you can read any regex pattern by decomposing it into its components, and you can write new patterns by composing those same components.
Think of metacharacters as the grammar of the regex language. Just as English grammar rules tell you how to combine words into meaningful sentences, metacharacter rules tell you how to combine pattern elements into meaningful descriptions of text structure. The grammar is small but recursive: you can nest groups inside quantifiers, combine character classes with anchors, and chain lookaheads with capturing groups. This composability is what gives regex its power.
The Dot: Match Any Character
The dot . matches any single character except a newline:
text = "cat cot cut c@t c9t c\nt"
# . matches any single character
pattern = r"c.t"
matches = re.findall(pattern, text)Text: 'cat cot cut c@t c9t c\nt' Pattern: c.t (c, any character, t) Matches: ['cat', 'cot', 'cut', 'c@t', 'c9t']
The dot matched 'a', 'o', 'u', '@', and '9', but not the newline. To match newlines too, use the re.DOTALL flag or the pattern [\s\S]. The dot is deceptively broad. In practice, using . in a pattern often causes unintended matches because it accepts characters you didn't think about. If you mean "any letter" you should write [a-zA-Z], not .. Reserve . for situations where you mean "any character," such as matching a fixed-width field of unknown content.
Character Classes: Matching Sets
Square brackets define a character class, matching any single character from the set. Character classes are one of the most important and frequently used features of regex, because real text rarely has a single accepted character in a given position. An American English name could start with any uppercase letter. A Unix username might contain letters, digits, hyphens, or underscores. A decimal number has digits in all positions. Character classes let you specify exactly which characters are acceptable at each position.
text = "The gray grey dog played in the fog"
# [ae] matches either 'a' or 'e'
pattern = r"gr[ae]y"
matches = re.findall(pattern, text)
# Ranges: [a-z] matches any lowercase letter
lowercase = re.findall(r"[a-z]+", "Hello World 123")
# Multiple ranges: [a-zA-Z0-9]
alphanumeric = re.findall(r"[a-zA-Z0-9]+", "user@example.com")Character class [ae]: Pattern: gr[ae]y Matches: ['gray', 'grey'] Range [a-z]: Text: 'Hello World 123' Matches: ['ello', 'orld'] Combined ranges [a-zA-Z0-9]: Text: 'user@example.com' Matches: ['user', 'example', 'com']
Negated Character Classes
A caret ^ at the start of a character class negates it, matching any character NOT in the set:
text = "abc123xyz"
# [^0-9] matches any non-digit
non_digits = re.findall(r"[^0-9]+", text)
# [^a-z] matches any non-lowercase letter
non_lower = re.findall(r"[^a-z]+", text)Text: 'abc123xyz' Non-digits [^0-9]: ['abc', 'xyz'] Non-lowercase [^a-z]: ['123']
Shorthand Character Classes
Regex provides convenient shortcuts for common character classes:
text = "Call 555-1234 or email bob@mail.com on 2024-01-15"
# \d = digit [0-9]
digits = re.findall(r"\d+", text)
# \w = word character [a-zA-Z0-9_]
words = re.findall(r"\w+", text)
# \s = whitespace [ \t\n\r\f\v]
spaces = re.findall(r"\s+", text)
# Uppercase versions are negations
# \D = non-digit, \W = non-word, \S = non-whitespace
non_digits = re.findall(r"\D+", text)Text: 'Call 555-1234 or email bob@mail.com on 2024-01-15' Shorthand classes: \d+ (digits): ['555', '1234', '2024', '01', '15'] \w+ (word chars): ['Call', '555', '1234', 'or', 'email', 'bob', 'mail', 'com', 'on', '2024', '01', '15'] \s+ (whitespace): [' ', ' ', ' ', ' ', ' ', ' '] \D+ (non-digit): ['Call ', '-', ' or email bob@mail.com on ', '-', '-']
The following table summarizes the most common regex character classes:
| Shorthand | Equivalent | Description |
|---|---|---|
\d | [0-9] | Any digit |
\D | [^0-9] | Any non-digit |
\w | [a-zA-Z0-9_] | Word character |
\W | [^a-zA-Z0-9_] | Non-word character |
\s | [ \t\n\r\f\v] | Whitespace |
\S | [^ \t\n\r\f\v] | Non-whitespace |
. | (any except \n) | Any character |
Note that uppercase versions match the complement (negation) of their lowercase counterparts.
The shorthand character classes deserve particular attention in NLP work. The \w class (word characters: letters, digits, and underscore) is a natural building block for matching tokens, but it has an important nuance: it matches the underscore character. This means \w+ matches hello_world as a single token, which may or may not be what you want depending on whether shows appear in your identifiers. The \s class matches all standard whitespace including newlines, which is useful when normalizing multi-line text into single-line tokens.