perf(rg): emit simple file matches as lines are searched

Security review details

🤖 auto-maintain review

Automated, advisory triage for @asafyish's PR. Facts below are read from the GitHub API.

Review panel: 🟡 medium highest severity

just-bash maintainer code review: 🟡 medium

The streaming work regresses established rg path-selection, diagnostics, glob precedence, and output compatibility.

• packages/just-bash/src/commands/rg/rg-search.ts:562 — Missing or unreadable paths are silently discarded, so `rg pattern /missing` now exits 1 without diagnostics instead of reporting the path and exiting 2.
• packages/just-bash/src/commands/rg/rg-search.ts:536 — Explicit file operands now pass through ignore/type/glob filters. Ripgrep searches explicit files regardless of recursive selection rules, so commands targeting ignored files can incorrectly produce no matches.
• packages/just-bash/src/commands/rg/rg-search.ts:790 — Gitignore and hidden checks occur before glob overrides, and every negative glob vetoes positives regardless of order. This breaks ripgrep's last-matching-glob semantics and prevents positive globs from re-including ignored or hidden paths.
• packages/just-bash/src/commands/rg/rg-search.ts:347 — Explicit `.` is no longer normalized to `./`, so results from `rg pattern .` lose their `./` filename prefix and differ from both ripgrep and the base behavior.

General code review: 🟢 low

No actionable correctness issues found in the complete diff.

Adversarial security: 🟢 low

No actionable adversarial security findings in the complete base-to-head diff.

Adversarial security (second opinion): 🟡 medium

No backdoors, exfiltration, dependency/CI tampering, or sandbox-escape paths found; the streaming stdio capability is properly revocable and limits remain enforced, but the default-sort traversal now double-charges the traversal/work budget and double-stats directory entries, which should be fixed or consciously accepted before merge.

• packages/just-bash/src/commands/rg/rg-search.ts:684 — The new `--sort path` key-building pass adds a second `budget.checkpoint()` for every directory entry on top of the existing one in the main loop (line 710). Since `sort: "path"` is the default, every recursive `rg` now consumes roughly 2x `maxTraversalWork` and 2x `executionScope.consumeWork` per entry, so searches fail with "filesystem traversal work limit exceeded" on trees about half the size they used to handle (noticeable at the 100k-work profile) and charge double against the shared work budget for the rest of the script. The same pass also issues an extra `ctx.fs.stat()` per entry when the filesystem has no `readdirWithFileTypes`, duplicating the stat the main loop performs anyway.

Standard Bash and host portability: 🟡 medium

The new streamed file matcher changes regex semantics for patterns that can consume line terminators.

• packages/just-bash/src/commands/rg/rg-file-lines.ts:39 — The streaming file path matches against `${line}\n`, unlike the buffered matcher, which removes line terminators before regex evaluation. Consequently patterns such as `\s$` match every newline-terminated line when `rg ... | <command>` is used, while ordinary ripgrep and the non-pipeline path only match lines ending in actual whitespace. Match the line content without exposing the synthetic newline, including for empty lines.

_{Posted by auto-maintain. This automated code review is advisory; a human maintainer makes the call.}

The timer finding is confirmed and fixed in cf06a8cb. A complete 80-line rg --no-line-number match /in | cat failed with SecurityViolationError before the change and now emits all 80 lines. The fix imports the existing captured _setTimeout; one regression added.

The concurrent-trust issue is also confirmed; 0b17670a carries the one-line dispatch fix and regression described on #453.

The claimed explicit-file/filter and missing-path regressions are pre-existing in the unchanged #453 base. I executed the examples on both revisions and posted the exact output/status table on #631: https://github.com/vercel-labs/just-bash/pull/631#issuecomment-6280966037 . Those compatibility bugs are outside this line-matching follow-up.

I also corrected an existing timing-dependent discovery test in a934b845: its listing now exceeds the 64 KiB pipe capacity, so cold module loading cannot let the whole fixture fit before the consumer starts. Both cases still fail on the non-streaming base and pass here. No runtime delays were added.

614 rg/pipeline tests pass (82 existing skips); type checking, formatting and banned-pattern checks pass.

Verified the latest review rather than applying its recommendations wholesale.

Fixed in 5d79c177:

• Work is now charged by cumulative characters plus lines, matching the buffered search's units instead of rounding every short line up to an extra unit. A 600-line file under a 1,000-unit matching budget reproduced the old streaming-only failure.
• Nonmatching lines no longer issue an empty pipeline write each. A first match followed by 6,000 nonmatches now completes under a 1,000-unit pipeline-work budget instead of exhausting it.
• The unconditional timer every 64 lines is replaced by periodic closure checks and elapsed-time-based yielding. Matching itself still uses the existing search helper; no new matcher or output-buffer mechanism was introduced.

The sparse-discovery issue is real and belongs to #631. Its fix searches the first file immediately, then resumes normal batches, following the existing grep pattern. Carried here in 1ac13655. Also carried the result-getter protection verified on #642 (5e751b82).

The remaining claims need qualification:

• Whole-tail buffering after a non-ASCII byte is intentional compatibility behavior in #638. For example, valid UTF-8 café followed later by byte 0xff makes the existing decoder interpret the entire input as Latin-1. Emitting the earlier line as UTF-8 would change its output. The existing mixed-encoding and BOM tests preserve that contract; incremental decoding requires a separately agreed behavior change.
• maxArrayElements bounds arrays; streaming stdin does not retain a line array. It already enforces matching work and charges the execution-wide work budget. I have not added a new line-count restriction to that path.
• Explicit-file filtering and missing-path diagnostics already reproduce on the unchanged #453 base. The concrete commands and results are in https://github.com/vercel-labs/just-bash/pull/631#issuecomment-6280966037. Those are existing compatibility issues, not regressions in this PR, and remain outside this fix.

The full rg suite plus pipeline checks passed: 618 tests, with 82 existing skips. Typecheck, formatting, and banned-pattern checks pass.

Verified the newline claim against actual streamed and buffered execution: with abc\ntrail \n\n, rg --no-line-number '\s$' /input | cat emits only trail \n; '^$' emits only the empty line. The claim is incorrect: searchContent removes line terminators before matching, including the synthetic one. No matcher rewrite is needed.

The real path-selection/diagnostic/glob/prefix regressions were resolved by rebasing the #631 prerequisite onto current main. The sorting pass now uses checkpoint(0) so it does not double-charge traversal work. On filesystems without readdirWithFileTypes, determining the directory sort key still needs metadata before traversal; the later lstat also identifies symlinks. I am retaining that bounded fallback rather than adding another metadata cache in this file-matching PR.

697 rg/pipeline checks pass (82 existing skips), plus typecheck, formatting, knip and banned-pattern checks.