Problem / Motivation
When the AI streams a reply in the WebUI, there is no way to see how fast it is generating. I'd like a live tokens-per-second indicator so I can tell whether the model is working normally or stalling.
Proposed Solution
While a reply is streaming, show a live tokens/sec readout near the message (e.g. under the reply or in the composer area), plus the total token count for that reply once it finishes. The gateway already tracks token usage server-side (llm_usage), so this is mostly a frontend display task.
Alternatives Considered
- Show only total tokens after completion (less useful for spotting stalls mid-stream).
- Show a generic "generating…" spinner (already exists, no speed info).
Related Component
Other (WebUI frontend)
Additional Context
Token usage is already recorded in the gateway's usage tracking; the WebUI just needs to surface it during streaming.
Activity
4
coinwh1mo agoFair point, a running tok/s number is mostly noise.
What I actually care about is a liveness signal. Right now the WebUI just says "generating…" and I can't tell these apart: the model is thinking, the provider stalled, or the connection is wedged and nothing will ever come. Sometimes it says "working" for minutes and nothing is actually happening — a rough indicator that bytes are still arriving (and one that stops when it's hung) would solve that for me.
chengyongru1mo agocoinwh1mo agoOne case I left out, and it's the one I actually hit: it doesn't always freeze. The output can drop from ~100 tok/s to ~1 — the timer keeps ticking, the spinner keeps spinning, no retry status ever fires, so it looks like a long think. And since the reasoning text is only expanded on hover, nobody's going to sit there watching it creep. A live number makes a 100× slowdown visible without hovering.
Does this have any particular significance? I tried doing it before, but it seemed to be nothing more than noise.🤔