Regular expressions (Python) | Programming Tutorials

The standard library tour introduced re.search, re.findall, groups and re.sub. This lesson takes regular expressions to the level you need for parsing logs, tokenizing input and validating text that comes from users: lookaround, verbose patterns, tokenizers, and the backtracking behaviour that can make one pattern freeze a server. Patterns are written as raw strings, r"...", so that a backslash reaches re unchanged.

Searching versus matching the whole string

re.search finds the first place the pattern matches and returns a Match object, or None. re.findall returns every match as a list of strings. re.fullmatch succeeds only if the pattern covers the entire string, which is what validation needs:

import re

text = "Order 1042 shipped, order 77 pending"
print(re.search(r"\d+", text))
print(re.findall(r"\d+", text))
print(re.fullmatch(r"\d+", "1042"))
print(re.fullmatch(r"\d+", "1042a"))

Output

<re.Match object; span=(6, 10), match='1042'>
['1042', '77']
<re.Match object; span=(0, 4), match='1042'>
None

fullmatch rejected "1042a" because the a is left over. Writing ^...$ with search is weaker: $ also matches before a trailing newline.

The pattern language, in brief

Piece	Matches
`\d`, `\w`, `\s`	a digit, a word character (letter, digit or `_`), a whitespace character
`[0-9a-f]`, `[^>]`	one character from a set, or one character not in it
`.`	any character except a newline
`*`, `+`, `?`, `{2,5}`	zero or more, one or more, zero or one, between 2 and 5 of the previous item
`^`, `$`, `\b`	the start and the end of the string, a word boundary
`a\|b`, `(?:...)`	either alternative, a group that does not capture

A backslash before a special character matches it literally: \. is a full stop.

Greedy and lazy quantifiers

Quantifiers are greedy: they take as much text as they can while still letting the whole pattern match. Adding ? makes one lazy, so it takes as little as it can:

import re

html = "<b>bold</b> and <i>italic</i>"
print(re.findall(r"<.+>", html))
print(re.findall(r"<.+?>", html))
print(re.findall(r"<[^>]+>", html))

Output

['<b>bold</b> and <i>italic</i>']
['<b>', '</b>', '<i>', '</i>']
['<b>', '</b>', '<i>', '</i>']

The third pattern says what it means, “a <, then anything that is not >”, and is usually the best choice. As the last section shows, it also cannot backtrack badly.

Groups and backreferences

Parentheses make a group. A Match object gives back what each group matched, numbered from 1, and group 0 is the whole match:

import re

line = "2026-10-12 14:03 ERROR disk full"
m = re.search(r"(\d{4})-(\d{2})-(\d{2})", line)
print(m.group(0), m.group(1))
print(m.groups())

Output

2026-10-12 2026
('2026', '10', '12')

Numbers become hard to follow beyond two or three groups. (?P<name>...) names a group, and the match can then be indexed by name or turned into a dictionary:

pattern = r"(?P<date>\S+) (?P<time>\S+) (?P<level>[A-Z]+) (?P<msg>.*)"
m = re.fullmatch(pattern, line)
print(m["level"], m["msg"])
print(m.groupdict())

Output

ERROR disk full
{'date': '2026-10-12', 'time': '14:03', 'level': 'ERROR', 'msg': 'disk full'}

When a pattern has groups, findall returns the groups instead of the whole match, as a tuple per match when there are several. finditer yields full Match objects, with positions:

prices = "tea 2.50, cake 3.75, water 1"
print(re.findall(r"(\w+) (\d+(?:\.\d+)?)", prices))
for m in re.finditer(r"(\w+) (\d+(?:\.\d+)?)", prices):
    print(m.group(1), m.span())

Output

[('tea', '2.50'), ('cake', '3.75'), ('water', '1')]
tea (0, 8)
cake (10, 19)
water (21, 28)

A backreference matches the same text a group already matched. (?P=name) refers to a named group inside the pattern, and \g<name> inserts it in a replacement. Together they remove doubled words:

doubled = "it is is a a test"
print(re.sub(r"\b(?P<word>\w+) (?P=word)\b", r"\g<word>", doubled))

Output

it is a test

Looking around without consuming

A lookaround checks what comes before or after the current position without making it part of the match. (?=...) is a lookahead, (?!...) a negative lookahead, (?<=...) a lookbehind and (?<!...) a negative lookbehind:

import re

bill = "tea $3.50, cake $4, 12 items, water €2"
print(re.findall(r"(?<=\$)\d+(?:\.\d+)?", bill))
print(re.findall(r"\d+(?= items)", bill))
print(re.sub(r"(?<=\d)(?=(?:\d{3})+$)", ",", "1234567"))


def strong(password):
    return re.fullmatch(r"(?=.*\d)(?=.*[A-Z])\S{8,}", password) is not None


print([strong(p) for p in ["hunter22", "Hunter22", "Hu2"]])

Output

['3.50', '4']
['12']
1,234,567
[False, True, False]

The amounts come back without the $. The third pattern matches no characters at all: it finds the positions that have a digit behind them and a multiple of three digits ahead, and sub inserts a comma at each one. In strong, each lookahead tests one rule from the start of the string, and \S{8,} then does the matching. A lookbehind must have a fixed width in re, so (?<=\$\d*) is an error.

Compiling, flags and verbose patterns

re.compile turns a pattern into a Pattern object with the same methods. Python caches recently used patterns, so compiling is less about speed than about naming a pattern once. Flags change how a pattern is read. re.IGNORECASE ignores case, re.MULTILINE makes ^ and $ match at every line, and re.VERBOSE ignores whitespace in the pattern and allows # comments:

import re

LOG = re.compile(
    r"""
    (?P<level>INFO|WARNING|ERROR)   # severity
    \s+
    \[(?P<module>\w+)\]             # module name in brackets
    \s+
    (?P<msg>.*)                     # the rest of the line
    """,
    re.VERBOSE | re.IGNORECASE,
)

for line in ["error [db] timeout", "INFO [web] started", "debug [x] y"]:
    m = LOG.fullmatch(line)
    if m:
        print(m["level"], m["module"], m["msg"])
    else:
        print("no match:", line)

Output

error db timeout
INFO web started
no match: debug [x] y

A tokenizer from named alternatives

One verbose pattern with a named group per kind of token, joined by |, turns finditer into a tokenizer, the first stage of any parser. m.lastgroup names the alternative that matched. A final . catches anything else, so no character is skipped silently:

import re

TOKEN = re.compile(
    r"""
    (?P<NUMBER>\d+(?:\.\d+)?)
  | (?P<NAME>[A-Za-z_]\w*)
  | (?P<OP>[-+*/()=])
  | (?P<SKIP>\s+)
  | (?P<ERROR>.)
    """,
    re.VERBOSE,
)


def tokens(text):
    for m in TOKEN.finditer(text):
        if m.lastgroup == "SKIP":
            continue
        if m.lastgroup == "ERROR":
            raise SyntaxError(f"unexpected {m.group()!r} at {m.start()}")
        yield m.lastgroup, m.group()

print(" ".join(f"{kind}:{text}" for kind, text in tokens("rate = 2.5*(x+10)")))
try:
    list(tokens("total = 3 $ 4"))
except SyntaxError as error:
    print("SyntaxError:", error)

Output

NAME:rate OP:= NUMBER:2.5 OP:* OP:( NAME:x OP:+ NUMBER:10 OP:)
SyntaxError: unexpected '$' at 10

Alternatives are tried left to right, so order matters: NUMBER comes before NAME, and a keyword group would have to come before NAME too.

Substitution

In a replacement string, \1 or \g<name> inserts what a group matched. The replacement can also be a function, which receives each Match and returns the text to put in its place:

import re

text = "Call 555-1234 or 555-9876."
print(re.sub(r"\d{3}-\d{4}", "XXX-XXXX", text))
print(re.sub(r"(\d{3})-(\d{4})", r"\2-\1", text))
print(re.sub(r"(?P<area>\d{3})-\d{4}", r"\g<area>-****", text))

Output

Call XXX-XXXX or XXX-XXXX.
Call 1234-555 or 9876-555.
Call 555-**** or 555-****.

def shout(m):
    return m.group(0).upper()


print(re.sub(r"\b[a-z]{2}\b", shout, "go to it or else"))

Output

GO TO IT OR else

Possessive quantifiers and atomic groups

When the rest of a pattern fails, re backtracks: it makes an earlier quantifier give back characters and tries again. Requires version 3.11 or later Python 3.11 added two ways to forbid that. A possessive quantifier, *+, ++, ?+ or {m,n}+, never gives back what it took. An atomic group, (?>...), does the same for a whole group:

import re

print(re.fullmatch(r"\d+5", "12345"))
print(re.fullmatch(r"\d++5", "12345"))
print(re.fullmatch(r"(?>\d+)5", "12345"))

Output

<re.Match object; span=(0, 5), match='12345'>
None
None

The greedy \d+ first takes all five digits, then gives back the 5 so that the literal 5 can match. The possessive and atomic versions keep all five, so the match fails. That is what you want when giving back can never help, as in the next section.

Backtracking can be exponential

Some patterns can match the same text in a huge number of ways. When the match then fails, re tries every one of them. (\w+\s?)* is the classic case: with no spaces in the input, a run of letters can be split between the repetitions of \w+ in 2^n-1 ways. On one laptop with 3.15, fullmatch on 24 letters followed by ! took about a second and a half, and each extra letter roughly doubles the time. A user who can send such a string can stall your program: a ReDoS (regular expression denial of service) attack.

The fix is a pattern in which every character can be matched in only one way, or, Requires version 3.11 or later with a possessive quantifier, one that refuses to backtrack:

import re

SLOW = re.compile(r"(\w+\s?)*")  # never run this on text you do not control
FAST = re.compile(r"\w+(?:\s\w+)*")
ATOMIC = re.compile(r"(?:\w+\s?)*+")

samples = ["two words", "one", "bad!"]
for pattern in (SLOW, FAST, ATOMIC):
    print([pattern.fullmatch(s) is not None for s in samples])

hostile = "a" * 100_000 + "!"
print(FAST.fullmatch(hostile), ATOMIC.fullmatch(hostile))

Output

[True, True, False]
[True, True, False]
[True, True, False]
None None

All three agree on ordinary input. FAST needs a space between two \w+ runs, so there is only one way to split the text. ATOMIC keeps the ambiguous body but never revisits it. Both reject 100,000 hostile characters at once. Warning signs are nested quantifiers such as (a+)+ or (\w+\s?)*, and alternatives that overlap, such as (a|a)*. The re module has no timeout. The third-party regex module offers one, and bindings to Google’s RE2 guarantee linear time by leaving out backreferences and lookaround.

prefixmatch and match

Matching only at the start of the string, without covering all of it, was long called re.match, a name that misled people: in most other languages match means what Python calls search.

Requires version 3.15 or later Python 3.15 adds re.prefixmatch and Pattern.prefixmatch as the clearer name and soft-deprecates match. Soft deprecation means the old name keeps working and will not be removed:

import re

number = re.compile(r"\d+")
for text in ["42 apples", "apples: 42"]:
    at_start = number.prefixmatch(text)
    anywhere = number.search(text)
    print(repr(text), at_start is not None, anywhere.group())

print(re.prefixmatch(r"\d+", "7 days").group())

Output

'42 apples' True 42
'apples: 42' False 42
7

For code that also runs on older versions, keep using re.match, or write re.search(r"\A...").

When not to use a regular expression

Use str methods for fixed text: startswith, in, split and replace are clearer and faster. Use a real parser for formats with nesting or quoting rules, such as HTML, JSON, CSV, dates and URLs: html.parser, json, csv, datetime.strptime and urllib.parse already handle the cases your pattern would miss. A pattern longer than a line or two without re.VERBOSE is a maintenance problem, and code that builds a pattern or a query out of user input belongs with template strings, which keep the parts apart.

Common pitfalls

search for validation. It accepts a valid piece anywhere in the string. Use fullmatch.
Forgetting the r prefix. In a normal string "\b" is a backspace character, so the pattern silently means something else.
Greedy .* across delimiters. Use a negated class such as [^"]*.
Calling .group() on None. When nothing matches, search returns None. Check the result first.
Nested quantifiers on untrusted input. (\w+\s?)* and (a+)+ backtrack exponentially. Make the pattern unambiguous, or possessive.

ExerciseA log-line parser

Write a compiled, verbose pattern with named groups for the lines below: a date, a time, a level, a module name in brackets and a message. Use fullmatch, print each line as time, level, module and message, and skip lines that do not fit. Expected output:

09:15:02 INFO auth: user ada logged in
09:15:07 ERROR db: connection lost (retry 1)
skipped: 'garbage line'
09:16:00 WARNING db: slow query: 2.5s

lines = [
    "2026-10-12 09:15:02 INFO  [auth] user ada logged in",
    "2026-10-12 09:15:07 ERROR [db] connection lost (retry 1)",
    "garbage line",
    "2026-10-12 09:16:00 WARNING [db] slow query: 2.5s",
]

Show solution

import re

LINE = re.compile(
    r"""
    (?P<date>\d{4}-\d{2}-\d{2})\s
    (?P<time>\d{2}:\d{2}:\d{2})\s
    (?P<level>[A-Z]+)\s+
    \[(?P<module>\w+)\]\s
    (?P<msg>.+)
    """,
    re.VERBOSE,
)

lines = [
    "2026-10-12 09:15:02 INFO  [auth] user ada logged in",
    "2026-10-12 09:15:07 ERROR [db] connection lost (retry 1)",
    "garbage line",
    "2026-10-12 09:16:00 WARNING [db] slow query: 2.5s",
]

for line in lines:
    m = LINE.fullmatch(line)
    if m is None:
        print("skipped:", repr(line))
        continue
    print(f"{m['time']} {m['level']} {m['module']}: {m['msg']}")

Output

09:15:02 INFO auth: user ada logged in
09:15:07 ERROR db: connection lost (retry 1)
skipped: 'garbage line'
09:16:00 WARNING db: slow query: 2.5s

In verbose mode spaces in the pattern are ignored, so each real space is written as \s, and \s+ after the level absorbs the extra space after INFO. (?P<msg>.+) is greedy, which is what you want for “the rest of the line”.

ExercisePredict: lookaround

What does this print? Remember that a lookaround matches a position, not characters.

import re

print(re.findall(r"\d+(?=px)", "10px 2em 30px"))
print(re.findall(r"(?<!-)\b\d+", "-5 7 -12 40"))
print(re.sub(r"(?=\d{3}$)", ".", "1999"))

Show solution

['10', '30']
['7', '40']
1.999

The lookahead requires px after the digits but leaves it out of the match, and 2 fails because em follows it. In the second line, 5 and the 1 of 12 have a minus sign before them. The 2 of 12 has no minus before it, but there is no word boundary between 1 and 2, so \b fails. The last pattern matches only the empty position that has exactly three digits after it, between 1 and 999, and sub inserts a full stop there.

ExerciseDefuse a slow validator

A form checks a list of comma-separated numbers with re.search(r"^(\d+,?)+$", text). A user sent 30 digits followed by x, and the check took minutes. Write is_number_list(text) with fullmatch and a pattern that can split the text in only one way. Empty items and a trailing comma are not allowed. For ["1,2,3", "10", "1,,2", "1,2,", "1" * 50_000 + "x"] it should print, at once:

[True, True, False, False, False]

Show solution

import re

NUMBER_LIST = re.compile(r"\d+(?:,\d+)*")


def is_number_list(text):
    return NUMBER_LIST.fullmatch(text) is not None


tests = ["1,2,3", "10", "1,,2", "1,2,", "1" * 50_000 + "x"]
print([is_number_list(t) for t in tests])

Output

[True, True, False, False, False]

In (\d+,?)+ the comma is optional, so a run of digits can be split between repetitions in exponentially many ways. In \d+(?:,\d+)*, every repetition after the first starts with a comma, so each digit belongs to exactly one place. The rewrite also rejects the trailing comma, which the original accepted. (?:\d+,?)++ would also be fast, but would keep accepting "1,2,".

Summary

search finds a match anywhere, fullmatch validates the whole string, and findall and finditer find every match.
Name groups with (?P<name>...), refer back with (?P=name) and \g<name>.
Lookahead and lookbehind check context without consuming it, and can match an empty position.
re.VERBOSE makes long patterns readable, and named alternatives with finditer and lastgroup make a tokenizer.
Requires version 3.11 or later Possessive quantifiers and atomic groups never give back what they matched.
Nested or overlapping quantifiers can backtrack exponentially. Write patterns that match each character in one way.
Requires version 3.15 or later prefixmatch is the new name for matching at the start, and match stays for older versions.
Next: algorithms the Pythonic way.