re.finditer() builds all matches up front and can hit the memory limit

I was looking through the regex implementation and noticed that re.finditer() / Pattern.finditer() currently build a list containing every match instead of returning a lazy iterator.

For example:

import re

p = re.compile(r"\d+")
it = p.finditer("a1 b22 c333")

print(iter(it) is it)
print(next(it).group())

In CPython:

True
1

In Monty, next(it) fails with:

TypeError: 'list' object is not an iterator

I traced this to RePattern::finditer(). It currently walks captures_iter(), creates every ReMatch, stores them in a Vec, and then returns a List.

Why this matters

Apart from the iterator behavior being different, this also affects memory usage.

For example:

import re

text = "a " * 100_000

it = iter(re.finditer(r"a", text))
result = next(it).group()

The code only asks for the first match, but with Monty's memory limits I get:

2 MB   -> MemoryError
4 MB   -> MemoryError
8 MB   -> MemoryError
16 MB  -> MemoryError
32 MB  -> succeeds

So all of the match objects appear to be created before the first result can be consumed.

The existing tests mostly call list(re.finditer(...)), which gives the same final result and hides this difference.

Possible approach

I think this could fit the existing iterator model by adding a small heap-backed regex iterator instead of returning a list.

The iterator would keep the state needed to continue the search, roughly:

pattern / compiled regex
subject
current search position
exhausted state

finditer() would return that object immediately.

Then each next() would:

1. search from the current position;
2. build one ReMatch;
3. update the search position;
4. return that match;
5. mark the iterator exhausted when there are no more matches.

This would also avoid trying to store the underlying Rust regex iterator directly, since that iterator borrows the regex and subject. Keeping an owned cursor/state object seems closer to how the other Monty iterators work.

One detail that would need care is zero-length matches so advancing the cursor continues to match CPython behavior.

The iterator would also need to participate in the normal heap/refcount/GC cleanup for whatever pattern and subject references it retains.

Tests

I think the regression coverage should include:

it = re.finditer(r"\d+", "a1 b22")

assert iter(it) is it
assert next(it).group() == "1"
assert next(it).group() == "22"

along with:

• no matches;
• capture groups;
• Pattern.finditer();
• zero-length matches;
• partial consumption;
• a memory-limit case showing that taking one match doesn't materialize the full result set.

I searched the existing issues and PRs for finditer / lazy regex iteration and couldn't find anything already tracking this.

If this direction makes sense, I'd be interested in working on it.