What
A viewer that can't keep up no longer slows everyone else down. Until now, every viewer (and every /record) shared one encoder: the bitrate followed the slowest viewer (RateCtl::update took the minimum), and flow_ok held captures back for whoever was furthest behind. Someone watching from a phone or bad Wi-Fi lowered the picture, and on a slow enough link the frame rate, for everyone.
Now such a viewer moves to a second, lower-bitrate stream. That stream only exists while someone needs it. Capture and conversion stay shared; each stream has its own encoder, rate control, flow control, paint-over and refinement state. A stream whose viewers are catching up skips captures and collects what changed (rows and macroblocks) until it can send again.
--slow-viewer-stream=false turns it off. One viewer costs exactly what it did before.
Results
16 vCPU VM, 2560x1600 Xvfb, 1080p animation (bench/loadgen), bench/nclient.py viewers. The slow viewer goes through bench/shaper.py with 60 ms RTT. Cells are FPS (worst second), latency p50/p99 and bitrate. Old and new builds were run interleaved.
| Case | Fast viewer, before | Fast viewer, after | Server CPU (cores) |
|---|---|---|---|
| Fast viewer alone | 60.0 (59.6), 10/16 ms, 7.21 Mbps | 60.0 (59.8), 10/13 ms, 7.24 Mbps | 1.04 → 1.02 |
| + a 5 Mbps viewer | 59.7 (57.9), 11/18 ms, 4.55 Mbps | 60.0 (59.7), 11/18 ms, 7.17 Mbps | 1.15 → 2.16 |
| + a 1.5 Mbps viewer | 23.1 (14.8), 14/28 ms, 1.29 Mbps | 59.9 (58.9), 11/24 ms, 7.15 Mbps | 0.56 → 1.48 |
| 5 Mbps viewer first, fast one joins 8 s later | 59.8 (56.9), 11/15 ms, 4.48 Mbps | 60.0 (59.9), 11/17 ms, 7.11 Mbps | 1.15 → 2.09 |
- The slow viewer gets the same throughput and frame rate as before (e.g. 4.51 → 4.59 Mbps at 5 Mbps, 22.9 → 22.6 FPS at 1.5 Mbps). Moving costs it one keyframe, a brief stall on its link: the worst second dropped to 41 FPS when the move happened mid-measurement.
- Recordings:
/recordstays on the main stream. A 15 s recording made while only a 3 Mbps viewer watched was 2.80 Mbps before and 7.00 Mbps now, the same quality as with a fast viewer. - CPU: the second encode costs about one core while it runs. With one viewer, or with viewers that all keep up, nothing changes.
- No false splits: two fast viewers, and a fast viewer with one on a 20 Mbps / 60 ms link, stayed on one stream.
How viewers move
Tiering in src/video.rs looks at the viewers every 100 ms:
- Out: a viewer's congestion (the check
RateCtlalready uses) adds to a decaying "strain" with a half-life of about 14 s. It moves to the slow stream once its strain reaches 3 while another main-stream viewer has been watched for 1 s without congesting. Congestion has to be remembered: once the shared stream has slowed down to suit a viewer, that viewer rarely congests again, and a fast viewer joining later would otherwise wait for the next probe (10–18 s in testing, versus ~1 s now). - The new stream starts just under the slow viewer's measured throughput (
RateCtl::seeded). The main stream forgets that viewer's capacity and ramps straight back up (RateCtl::reprobe). - Back: a viewer rejoins once it has gone 10 s without congestion and has received at least 80% of what the main stream's viewers get, measured while the main stream carries more than the bitrate floor. The 10 s doubles with every move (up to 16x), so a borderline link settles instead of flapping. If the main stream's viewers all leave, the slow viewers take it over with their own rate controller, and the second encoder stops.
- Switching streams:
Client::move_toand the encoders' sends both run under the viewer'sinflightlock, so nothing from the old stream can follow the new stream's keyframe. Frame IDs come from one counter (Hub::frame_id), so a viewer never sees an ID twice. The web client needs no changes: it already starts over at any keyframe.
Smaller changes:
- The two capture buffers are shared as
Arc<I420>; a buffer is reused once every encoder that took it has handed it back. - Encoders open on their own thread, so starting the slow stream doesn't stall the capture loop.
/statsaddsslow_target_kbpsand each client'stier.- The README gains the flag and a paragraph.
Testing
cargo fmt --check,cargo clippy --all-targets --locked -- -D warningsandcargo testwith CI's nixpkgs toolchain;npm run check.- New unit tests for
Tiering:- the viewer holding the others back moves, after the other viewer has been watched for a while;
- nobody moves when alone, when everyone is congested, or with the flag off;
- rejoining needs both the calm period and keeping pace;
- slow viewers take over the main stream when the others leave.
- End to end, with fast and shaped viewers: the cases above, plus:
- a 3 Mbps link that recovers after 20 s: the viewer moves back ~17 s later, once;
- a link that never recovers: it stays put for 60 s;
- the fast viewer leaving;
/recordwith a slow viewer;--stripes=2;- a dynamic-resolution resize while a viewer is on the slow stream: both viewers get the new size and keep streaming;
--slow-viewer-stream=false, which behaves like before.
- The real web client in headless Chromium, through a 3 Mbps link, with a fast viewer next to it: it decodes the slow stream at 58 FPS / 3 Mbps, then switches back to the main stream's encoder when the fast viewer leaves, with no decoder errors.
Not tested on macOS or Windows. The change is in the platform-neutral src/video.rs and src/hub.rs.
Want tembo to make any changes? Add a comment with
@temboand i'll get back to work!