#626

perf(search): skip text without a pattern's required literals in rg and grep

Openasafyishwants to merge intofrom

Work out the literals every match of a pattern must contain (product.*update needs both product and update), and skip files and lines without them before RE2 runs. rg now uses this filter too. Until now only grep had one, and only for patterns made of plain literals joined by |.

Fixes #625.

The repro from #625 (20 MB, Node 26.8.2, Apple Silicon):

mainthis PR
rg -c "Target_Date__c|VP_MC__c"4380 ms161 ms
rg -c "Target_Date__c"299 ms130 ms
rg -c -i "target_date__c"7274 ms138 ms
rg -c "inactive|product.*update"9448 ms147 ms

The analysis is conservative. A construct it doesn't model (inline flags, lookaround, an ambiguous brace, an alphanumeric escape) turns the filter off for that pattern, so it only skips text the regex can't match.

Under -i, needles are lowercase ASCII, and text is lowercased with ſ mapped to s. An exhaustive scan with re2js shows that ſ (U+017F) and K (U+212A) are the only non-ASCII code points RE2 folds into a–z, and toLowerCase already maps K. The old filter missed ſ, so grep -i sap dropped a line containing ſap.

Most of the test diff updates regex.test.ts to the new groups shape. Several patterns it listed as unsafe to extract (a+, (foo), \bfoo, a.b) now yield a sound requirement. required-literals.test.ts covers the analysis and the folding, and rg.prefilter.test.ts pins rg output where a wrong requirement would drop a match.

Validation: lint, typecheck, and the rg, grep and search-engine tests with their comparison tests pass.

Activity

3
vercel[bot]14d ago

@asafyish is attempting to deploy a commit to the Vercel Labs Team on Vercel.

A member of the Team first needs to authorize it.

vercel-security-reviewer[bot]14d ago