perf(chat): keep untouched rows and parts still while a turn streams

Why

Replaying one real session's 1,803 recorded events (13 turns, mostly tool calls and reasoning) into the production web build costs 5.7 ms of script time per event, rising to 9.5 ms as the thread grows. Render counts from the dev build for 150 of those events show where it goes: the composer (MessageInputView, ModelSettingsControl, ProviderModelPicker) and the permission/question/plan providers rendered once per event, and every ActivityGroup / ToolCallPart of the live turn rendered again on each event (1,725 and 547 renders).

All of it has one root: every streamed event rebuilt the whole live turn. projectTurn returned new part objects for every thought and tool, a new UIMessage per row, and so a new messages array, which made the ActiveThreadStateContext value new, which re-rendered everything that reads it for its commands.

What changed

• Stable parts. The projection keeps one part per store entry (WeakMap on the ToolState / ReasoningEntry / Message object). The store replaces only the entry an event touched, so every other part of the turn is the same object it was.
• Stable rows. The assistant row's UIMessage and the row of a waiting send keep their identity while they say the same thing, so projection.messages (and with it the thread-state context value) only changes when a row is added, confirmed or settles.
• Per-turn split once per list instead of filtering all four thread lists for every turn on every event; the order walk looks entries up by id instead of scanning.
• Memoized part nodes. MessageParts wraps each part and each tool-call run in a memoized node; a run is compared by its members and summary fields, so an event re-renders the part it changed and nothing else.
• The interaction providers read only the active thread id, not the whole thread.
• Timeline keeps following the bottom (commits 3–5, see below).

Nothing about the contracts changed; desktop's providers are untouched.

Production build: script time per replay pass

Same 1,803 events replayed four times into the same session (headless Chrome, production bundle, 4 ms between events). Script time per pass:

Before, the cost grows with the thread (every event re-rendered everything that had ever streamed); after, it stays roughly flat. Renderer memory is unchanged by the PR (≈125–147 MB after four passes either way: the 1 GB+ tab reported on the dev server is React's dev-only performance.measure entries, not something this PR addresses).

Before / after side by side (first 400 events, 16 ms apart, at 4× CPU throttling so the difference shows at normal speed; the clock in the corner is wall time inside the page — the before build falls behind the event stream, the after build keeps up):

Full recordings: replay-before.webm · replay-after.webm. Stills at 20 s: before · after. (Assets live on the assets/pr-188-stream-render branch, not in the app; delete after merge.)

Render counts, dev build, 150 streamed events

What still renders per event is the live assistant row itself (AssistantMessage, MessageParts) and the running tool group whose summary line changes. The sidebar renders once per turn (workspace lastActivityAt moves on turn.started); not touched here.

The scroll-follow bug this surfaced (commits 3–5)

Measuring the first two commits showed the timeline sometimes stopped following the bottom mid-stream (gap of thousands of px, 25–29 rows mounted instead of 9–13). Traced with a probe that hooks the virtualizer through the React fiber; three independent causes, all in code this PR did not touch but whose timing it changed (the third is pre-existing on main and only reachable in a tab the browser has stopped painting):

1. Append race. TanStack's followOnAppend decides "was at the end" with its DOM-based isAtEnd() (scrollHeight − clientHeight vs its internal offset). When a row is appended in the window after the streaming row grew in the DOM but before the ResizeObserver reported it (6 px in the captured trace: count 35→37 atEnd:false sh=13954 listH=13948 off=13084), the append is not followed, and every later resize finds the end further away still. The timeline now judges "at the end" by the virtualizer's own self-consistent numbers before the append (snapshotted once per row count, so StrictMode's second render does not see the new rows), checks the scroller's own position against that bottom so a reader who just wheeled up is never dragged, and calls scrollToEnd() itself when the row count grew — also while a previous follow is still pending, since the virtualizer only learns the new offset from the scroll event a frame later (two appends in one frame, or a background tab).
2. Chrome scroll anchoring. With the default overflow-anchor: auto, hiding 21–46 px above the viewport inside a row moved scrollTop by the same amount with no script involved; a tool group settling does the same during a stream, and that stray scroll reads as the reader leaving the bottom. The scroller now opts out ([overflow-anchor:none]) only while following the bottom; a reader who has scrolled back keeps the browser's anchoring, so what they are reading holds still when a row above them in the same reply changes height (the virtualizer deliberately skips that compensation).
3. Stale scroll-end replay. With the default useScrollendEvent: false, virtual-core decides "scrolling stopped" with a 150 ms timer that replays the offset captured at the last scroll event (observeOffset), not the live scrollTop. In a tab the browser has stopped painting (hidden, or behind another tab) scroll events arrive about once a second while the virtualizer's own resize adjustments keep moving its offset in between, so the timer reverts them — in the trace, 9145 → 8682, exactly the −70 + 533 it had just applied — and the next append reads as arriving 463 px above the end. A setter trap on the offset showed the replay firing 150 ms after every adjustment cycle; runs only passed when no append landed inside the stale window. The timeline now passes its own observeElementOffset: the library's observer verbatim, except the timer reads the scroller. (useScrollendEvent: true was tried first and reviewed out: scrolls with no scrollend — a clamp when content below shrinks, a scroll-anchoring shift — would leave the virtualizer's isScrolling/scrollDirection stuck, which disables its resize compensation for a reader who has scrolled up; and it does nothing on browsers without scrollend.)

Verification (none of this is unit-testable; jsdom has no layout), all against the built bundle in headless Chrome:

• a probe that pads the live row by 4 px right before every turn.started forces the race deterministically — gap 1,910 px at the end before the fix, 0 after;
• a reader who wheels up mid-stream drifts 0 px through ten appends and keeps the scroll-to-latest button;
• a 21 px shrink above the viewport inside the live row no longer moves the scroller while following;
• the two-pass replay that reproduced the original failure (gap 3,042 / 11,447 px) now ends at 0 / 0 with 13 rows mounted, and the four-pass CPU run keeps 13 rows and 1,336 DOM nodes throughout (it had drifted to 29 rows / 6,752 nodes when the view was stuck);
• a chat tab hidden behind another while a whole reply streams in (events dispatched in bursts from outside the page, since hidden tabs throttle timers; rAF ran at 1/s) comes back at the bottom: before commit 5 it detached by 5,841 px in 2 of 5 runs, after it stays at the bottom in 3 of 3 runs, with zero stale offset writes (17–25 per run before).

Tests

• environment-thread.test.ts: parts and the message list survive a streamed event; the row gets a new identity when the turn settles; a waiting send keeps its row.
• tests/stream-render.test.tsx: the thread contract is not handed to its readers again for tokens and tool calls (a stand-in for the composer) but is when the turn completes; MessageParts re-renders only the part an event changed, and leaves a whole run alone when the event lands outside it.

Reviews

Commits 1–2: independent reviews by Codex (gpt-6.1-sol, high) and Opus — both mergeable, no findings; Greptile: no actionable issue. None caught the follow bug (a frame-boundary race; the probe did). Commit 3 was reviewed the same way: Opus and Codex each returned "mergeable with fixes" with the same findings (follow through consecutive appends, StrictMode double render, reader-position check, anchoring for a reader scrolled up); Greptile raised the last one as a P1. All four are in commit 4, re-verified with the probes above.

Commit 4 was reviewed again by Opus, which found three further timing holes of the same shape: a row re-measured synchronously in a background tab after a pending follow was not followed; TanStack's own followOnAppend makes the same stale read as cause 1 and fires before the guarded effect, so it could still drag a reader who had just scrolled up; and the anchoring class dropped for a frame on every append. Commit 5 closes them (onChange honours a pending follow by the scroller's own position, followOnAppend: false so the guarded effect is the only append follow, at-end reported from the follow itself) and was reviewed by Opus (mergeable) and Codex gpt-6.1-sol (mergeable with one fix); their two one-line findings — skip the virtualizer's above-viewport resize compensation while a follow is pending, since its offset is stale then, and let the footer path honour a pending follow — are folded in. The background-tab probe Opus asked for then found cause 3 (commit 5, as useScrollendEvent: true); Opus reviewed that option and measured the stale replay in a visible tab too (20 of 82 adjustment cycles reverted with the default observer, 0 with the fix), but found the stuck-isScrolling downside above, so commit 6 replaces it with the custom observer, which keeps the library's timing exactly.

🤖 Generated with Claude Code