feat(rg): stream discovery and results with downstream cancellation

🤖 auto-maintain review

Automated, advisory triage for @asafyish's PR. Facts below are read from the GitHub API.

Review panel: 🟡 medium highest severity

just-bash maintainer code review: 🟡 medium

The streaming implementation introduces observable rg correctness regressions and does not provide downstream cancellation for stdin searches.

• packages/just-bash/src/commands/rg/rg-search.ts:278 — Stdin search calls readCommandStdin/readAll before matching, so `producer | rg needle | head -1` cannot emit or cancel upstream until EOF; large or infinite producers instead stall or hit maxInputBytes.
• packages/just-bash/src/commands/rg/rg-search.ts:529 — Explicit file operands now pass through ignore/type/glob filtering. Commands such as `rg needle ignored.txt` can silently skip a directly named file, contrary to ripgrep semantics.
• packages/just-bash/src/commands/rg/rg-search.ts:555 — Discovery discards path-access errors, making `rg needle /missing` return a silent no-match status instead of the existing diagnostic and error status.

General code review: 🟡 medium

The streaming implementation introduces observable rg operand regressions and does not implement its claimed stdin cancellation path.

• packages/just-bash/src/commands/rg/rg-search.ts:529 — Explicit file operands now pass through ignore/type/glob filtering. Commands such as `rg -g '*.js' pattern explicit.txt` incorrectly skip the explicitly requested file, unlike prior and ripgrep behavior.
• packages/just-bash/src/commands/rg/rg-search.ts:553 — Stat failures for requested paths are silently discarded. `rg pattern /missing` now returns an undiagnosed no-match, and mixed valid/missing operands can also suppress the valid file's filename prefix.
• packages/just-bash/src/commands/rg/rg-search.ts:278 — Piped stdin is still fully collected by `readCommandStdin` before matching, so pipelines such as `rg --files | rg pattern | head` cannot propagate early downstream closure and may hit buffered-input limits despite the release note claiming line-oriented stdin streaming.

Adversarial security: 🟢 low

No actionable adversarial security findings in the complete base-to-head diff.

Adversarial security (second opinion): 🟡 medium

No backdoors, exfiltration, dependency tampering, or sandbox weakening; the new stdio capability is properly revocable and limit-accounted. Two actionable issues: the advertised line-oriented stdin streaming for rg is not implemented, and the new empty-pipe stdin rule diverges between the streaming and buffered pipeline executors.

• packages/just-bash/src/commands/rg/rg-search.ts:278 — The PR body and .changeset/rg-streaming.md both claim line-oriented stdin streaming with downstream closure "including ordinary `rg --files | rg pattern | head` pipelines", but the stdin search path unconditionally calls readCommandStdin(ctx) -> stdio.readAll(), which drains upstream to EOF before matching. In that exact example the middle rg blocks until `rg --files` has enumerated the whole tree, so head cannot stop the producer and the advertised fix does not apply. Either implement the prefix streaming or correct the changeset/description before it ships to users.
• packages/just-bash/src/commands/rg/rg-search.ts:282 — Adding `ctx.stdio?.hasInput ||` makes `rg PATTERN` with no path operands search an empty upstream pipe instead of falling back to the cwd, but only inside streaming-eligible pipelines; the buffered executor still keys off `stdinText.length > 0`. The same script therefore diverges on an opaque eligibility heuristic: `cat /dev/null | rg SECRET` now searches nothing (exit 1), while `true | rg SECRET` (builtin stage, so the pipeline is disqualified) still recursively searches the entire working tree. That is a materially different disclosure surface for identical-looking code. Apply the hasInput rule to both executors, or gate it so the two paths agree.

Standard Bash and host portability: 🟡 medium

The file-search path streams, but the promised line-oriented stdin pipeline remains non-streaming.

• packages/just-bash/src/commands/rg/rg-search.ts:278 — Piped stdin is still fully buffered before searching. Awaiting `readCommandStdin(ctx)` consumes the producer to EOF, so pipelines such as `rg --files | rg pattern | head` cannot propagate downstream cancellation and may traverse everything or hit input limits before emitting a match.

_{Posted by auto-maintain. This automated code review is advisory; a human maintainer makes the call.}

I checked the regression claims by executing the same examples on the unchanged #453 base (262a17b2) and this PR. With /src/a.txt containing needle\n, both revisions produce:

The explicit-file filtering, missing-path diagnostics, and mixed-operand prefix behavior are existing compatibility bugs, not regressions introduced by this PR. Native ripgrep does behave differently. I am keeping those separate from this file-discovery/streaming change rather than expanding it into unrelated rg semantics.

The stdin-streaming criticism is correct about the stale changeset, which is now corrected in 6898f9ab. The implementation and tests were deliberately split into #638; this PR's description already identifies that split. Stdin matching here remains whole-input.

The empty-pipe discrepancy also needs qualification: the base searches the cwd for both true | rg needle and cat /dev/null | rg needle. This PR fixes the latter when an actual upstream pipe is identifiable (hasInput); it does not introduce the buffered executor's inability to distinguish absent input from empty input. Fixing that remaining case needs buffered-runtime input provenance. I have retained the correct empty-stream behavior rather than reverting it to searching files.

Also carried the shared trust-scope fix from #453 in e73c01b4. The focused rg/pipeline run passed 596 tests with 82 existing skips; one traversal/startup timing assertion failed once and its focused eight-test file passed on recheck with and without the trust change. No rg matching code changed in this follow-up.

Follow-up: 9dc66a91 removes the startup-timing assumption from the discovery regression. The listing now exceeds the 64 KiB pipe capacity; both early-stop cases still fail against the unchanged #453 base and pass here. All eight discovery tests pass. This changes only the existing fixture, with no runtime scheduling changes.

Verified the sparse-discovery finding reported on dependent #639. Before 79c7d92d, a matching first .json file was held until discovery exhausted later directories trying to fill a 50-file batch. The new regression confirms that traversal of /src/z is unnecessary for rg -g '*.json' needle /src | head -n 1 when /src/a.json already supplies the result.

The fix follows grep's existing pattern: search a one-file first batch, then retain the normal concurrent batch size. One focused regression was added, and the existing concurrent-read test still verifies bounded read-ahead after the first file. The same fix is carried into #638 and #639.

Also carried the shared result-getter protection from #642 in 2db7c8fd. 68 focused discovery and pipeline tests and typecheck pass.

Rebased onto current main, preserving #584’s explicit-file selection, path diagnostics, glob overrides and ./ prefix, plus #401’s ownership of empty piped stdin. Fixed the real double traversal charge with the existing checkpoint(0) pattern. Also fixed quiet --files-without-match: a hit followed by a miss now exits 0 without accumulating filenames. Two focused regressions cover these. Line-oriented stdin remains deliberately separate in #638; the description and changeset identify that boundary.

Security review details