Three separate problems, all of them invisible on the first test anybody writes.
text = 'Cat and category'
text.replace('cat', '**cat**')
# -> 'Cat and **cat**egory' 'Cat' missed, and the case is now wrong
# Loop over ['cat', 'category'] and the first pass has already
# broken the word the second one was looking for.
import re
pattern = re.compile('category|cat', re.IGNORECASE) # longest first
pattern.sub(lambda m: '**' + m.group(0) + '**', text)
# -> '**Cat** and **category**'
- m.group(0) is how the original case survives
The replacement is built from what was actually matched, not from the term that was searched for — so 'Cat' comes back as 'Cat' even though the term was 'cat'. That is the whole reason for passing a function to sub instead of a replacement string: the function gets the match object and can use what it found.
- Alternation takes the first branch that matches, not the longest
Python’s regex engine tries the alternatives left to right and stops at the first success, so 'cat|category' matches only 'cat' inside 'category'. Sorting the terms longest-first before joining them makes the more specific term win. This is a genuine difference from some other engines, and it catches people out regularly.
- One pass cannot highlight its own markers
sub scans the string once and continues after each replacement, so the asterisks it inserts are never looked at again. A loop of replace calls re-reads the whole string every time, which is how a term that appears inside your own marker — or inside a previously highlighted word — ends up wrapped twice.
Build one pattern from all the terms and do a single substitution. The replacement has to be a function, because it needs to see what was actually matched.
Drop the empty terms, sort what is left longest first, escape each one, and join them with a pipe. Compile with re.IGNORECASE. Handle 'nothing left after filtering' by returning the text before you build anything.
re.escape(term) makes a term literal; '|'.join(...) makes the alternation; sorted(usable, key=len, reverse=True) makes the longest win. The replacement is lambda match: marker + match.group(0) + marker — group(0) is the matched text with its original case.
import re def highlight_terms(text, terms, marker='**'): usable = [term for term in terms if ____] if not usable: return ____ ordered = ____(usable, key=len, reverse=True) pattern = re.compile( '|'.join(re.____(term) for term in ordered), re.____, ) return pattern.sub(lambda match: marker + match.____(0) + marker, text)
import re
def highlight_terms(text, terms, marker="**"):
usable = [term for term in terms if term]
if not usable:
return text
# Longest first: alternation takes the first branch that matches,
# not the best one, so 'cat|category' would never match 'category'.
ordered = sorted(usable, key=len, reverse=True)
pattern = re.compile(
"|".join(re.escape(term) for term in ordered),
re.IGNORECASE,
)
return pattern.sub(lambda match: marker + match.group(0) + marker, text)
The terms usually arrive as the caller’s own list — a parsed query, a list of keywords held elsewhere — and sorting it in place reorders something they still own for reasons they will never guess. It is the same rule as everywhere else: a function that answers a question should not rearrange its arguments.
Terms come from users, and users search for 'C++', '1.2', 'what?' and '(draft)'. Without escaping, those are patterns rather than text: at best they match the wrong thing, at worst they fail to compile and take the page down. Any time user input reaches a regular expression, it goes through re.escape first.
Both an empty list and a list of empty strings have to reach the same early return, because an empty alternative matches at every position and would wrap the space between every pair of characters. Removing the empties first means one check covers both, and the pattern that gets built can never contain a branch that matches nothing.
If the output is HTML, this is not enough on its own — the marker has to be a tag and the surrounding text has to be escaped, in that order, or a search for '<script>' becomes an injection. If a real search engine is involved, let it do the highlighting: Elasticsearch, Postgres full-text search and Lucene all return highlighted fragments and know about stemming, so 'running' can highlight 'run'. This function matches literal substrings, which means it also highlights matches inside longer words — add \b around the alternation if you want whole words only, and be ready for what that does to terms containing punctuation.
- Wrap only whole words by putting word boundaries around the alternation, and find out what happens to a term like 'C++'.
- Return the number of matches alongside the text, so a caller can say 'no results found for that word'.
- Escape the text for HTML and use a <mark> tag as the marker, getting the order of the two operations right.