Summary
PCM becomes a fourth audio format, carried over the existing audio codec offer/accept mechanism. The default stays opus.
DESCRIBE
codec_offer(audio_format, ac3_available, eac3_available) # new, pure
pcm -> "PCM" # no encoder needed
ANNOUNCE
configure_codec(offered, accepted, ...) # was configure_dolby_audio
offered == "PCM" && accepted == offered
use_pcm = true
packetDuration = 5 ms (client value ignored)
HIGH_QUALITY, CUSTOM_SURROUND_PARAMS = off
reply X-SS-Audio-Codec: PCM
otherwise -> Opus audio, no error
video bitrate -= pcm_bitrate_kbps(channels) # 20% cap; PyroWave x1.5, uncapped
audio encode thread
if use_pcm
skip Opus/FFmpeg
payload = pcm_payload(samples) # *32768, round, clamp, int16 LE, interleaved
raise packet; forward samples to WebRTC as the other paths do src/
├── audio.h/.cpp # codec_e (+pcm), configure_codec, codec_name, codec_offer, pcm_bitrate_kbps
├── audio_policy.h/.cpp # + pcm_payload
├── rtsp.cpp # offer via codec_offer; PCM never refused; bitrate adjustment
├── config.cpp # audio_format accepts "pcm"
└── web UI + docs # dropdown option, English strings, bandwidth notesOther changes:
high_quality_audiois now visible only forac3andeac3, via a newoneOfvisibility condition.- Docs and English strings describe PCM and its bandwidth (about 1.5, 4.6 and 6.1 Mbps for stereo, 5.1 and 7.1).
Evidence
These tests are added by the PR.
- Before:
PCMoffered and accepted has no handling.configure_dolby_audioreturns Opus, andaudio_format = pcmis rejected by config validation. After (ANNOUNCE composition, 8 rows → 20):PCM/PCM stereo, 5.1, 7.1 (client sent 10 ms or none) -> 200, "PCM", use_pcm, 5 ms, Opus flags off PCM offered, nothing accepted -> "Opus" PCM offered, EAC3 accepted -> "Opus" AC3 offered, PCM accepted -> "Opus" bitrate: 2ch 97964, 6ch 94892, 8ch 93356 (of 100000) cap: 7.1 at 10000 kbps -> 7500 PyroWave: 2ch 97196, 8ch 90284; 7.1 at 9000 kbps -> 400 existing AC3/EAC3 rows unchanged - Before: no PCM conversion exists.
After (
AudioPcmPayload, no FFmpeg or hardware needed):pcm_payload({4660/32768, -4660/32768, 0}) == 34 12 CC ED 00 00 # little-endian, interleaved rounds half away from zero: 1.5 -> 2, -1.5 -> -2 clamps 1.0, 1.5, 1e10 -> 0x7FFF; -1.0, -1.5, -1e10 -> 0x8000 5 ms at 48 kHz -> 960 / 2880 / 3840 bytes for stereo / 5.1 / 7.1 - After (UI):
high_quality_audio.visibleWhen == { key: 'audio_format', oneOf: ['ac3','eac3'] }
Merge Danger
Door: two-way
PCM is opt-in. The audio format defaults to opus, so nothing changes until a user selects pcm. Reverting the PR leaves no persisted state behind. The one exception is a config that already says audio_format = pcm, which a revert would reject as an invalid value.
Blast Radius: negotiation
- Shared negotiation code: the PR renames
dolby_codec_e,configure_dolby_audioanddolby_codec_name, and moves the offer choice intocodec_offer. The existing Dolby test rows are the guard against regressions. The branch is 5 commits behindmaster, so the rename may conflict with other audio work. - PCM sessions: PCM packets for 5.1 and 7.1 (2880 and 3840 B) rely on IP fragmentation. Losing any fragment loses the whole packet, so lossy networks get more silence gaps. PCM also reserves up to 6.1 Mbps from the video bitrate.
- Unverified: only a PCM-capable client (Artemis) can exercise the PCM path. The audio encryption path is untouched and has no PCM-specific test.
Problem Statement
Vibepollo sends audio to Artemis as Opus audio, or as Dolby audio (AC-3 / E-AC-3) when the audio format selects it and the client accepts it. Both are lossy, so music, soundtracks and positional surround cues arrive with compression artifacts even on networks with spare bandwidth. Dolby audio also only covers stereo and 5.1, so there is no way to get the host's audio bit-for-bit in any channel layout, 7.1 included.
Solution
Make PCM audio a fourth choice of the audio format (pcm), carried over the existing codec offer/accept mechanism. With the audio format on PCM, DESCRIBE offers PCM. A client that accepts it in ANNOUNCE gets 16-bit signed, 48 kHz audio in stereo, 5.1 or 7.1 in 5 ms packets. A client that doesn't accept it silently gets Opus audio, as today. The default stays opus, and Dolby behaviour is unchanged.
DESCRIBE a=x-ss-audio-codec-offer:PCM (audio format = pcm)
ANNOUNCE x-ss-audio-codec-accept:PCM
reply X-SS-Audio-Codec: PCM
audio encode thread skips Opus/FFmpeg -> int16 LE payload, 5 ms packetsImplementation Decisions
- Config / UI / docs:
audio_formatacceptspcm(config.cpp). The web UI dropdown gets a PCM option with English strings only (settingsSchema.ts,en.json).docs/configuration.mddocuments PCM audio, its per-layout bandwidth (about 1.5 / 4.6 / 6.1 Mbps) and the high quality audio note. - Offer choice extracted:
audio::codec_offer(format, ac3_available, eac3_available)is a pure function used by DESCRIBE.pcmalways offersPCMbecause no encoder is needed. - Dolby-specific names generalized:
dolby_codec_e/configure_dolby_audio/dolby_codec_namebecomecodec_e(opus, ac3, eac3, pcm) /configure_codec/codec_name.config_tgainsuse_pcmbesideuse_ac3/use_eac3, which keeps the diff against upstream small. - Negotiation: PCM is selected only on an exact offer/accept match. The packet duration is fixed to
PCM_PACKET_DURATION(5 ms) regardless of the client'sx-nv-aqos.packetDuration. TheHIGH_QUALITYandCUSTOM_SURROUND_PARAMSflags are turned off, and the high quality audio setting is ignored. ANNOUNCE can only refuse (400) a Dolby accept the host can't provide. PCM supports every channel layout, so it is never refused for that reason. - Capture path: when
use_pcmis set, the encode thread skips the Opus and FFmpeg encoders.policy::pcm_payloadscales each float sample by 32768, rounds to nearest, clamps, with no dither, and writes little-endian int16 in interleaved order (audio_policy.cpp). Samples are forwarded to WebRTC under the same condition as the other paths. - Bitrate budget:
audio::pcm_bitrate_kbps(48000 × 16 × channels = 1536 / 4608 / 6144 kbps) feeds the existing video bitrate adjustment, and the existing 20% cap and PyroWave rules (× 1.5, uncapped, session refused if it doesn't fit) apply unchanged. - ANNOUNCE reply:
X-SS-Audio-CodecreportsPCMviacodec_name. - Unchanged: the 4096-byte audio packet limit, audio encryption, and 4+2 Reed-Solomon recovery.
- Related UI fix (not in the original issue):
high_quality_audiois now shown only forac3/eac3, using a newoneOfcondition in the settings schema andSettingsView.vue, since the setting has no effect for Opus or PCM.
Testing Decisions
- ANNOUNCE composition test (
test_host_stream_terminal_composition.py,composition.cpp.in), which compiles the real ANNOUNCE handler with the real negotiation code. The fixture now follows the renamed functions, includescodec_offerandpcm_bitrate_kbps, and records the adjusted video bitrate (allocated_bitrate). The table grows from 8 to 20 rows, and the existing Dolby rows keep passing and gain bitrate expectations.- PCM accepted in stereo / 5.1 / 7.1 gives PCM, 5 ms (client-sent 10 ms or none), and Opus-only flags off.
- PCM offered but not accepted,
PCMoffer withEAC3accepted, orAC3offer withPCMaccepted all give Opus. - The high quality audio setting doesn't change a PCM session.
- The bitrate reduction is checked as capped at 20% (7500 of 10000 kbps at 7.1), and PyroWave is checked as reserved in full (× 1.5) or refused with a 400 at 9000 kbps.
codec_offeris checked foropus,ac3,eac3andpcm, andpcm_bitrate_kbpsfor 1536 / 4608 / 6144.
- Audio policy unit tests (
tests/unit/test_audio.cpp) coverpcm_payload: little-endian int16 in interleaved order, round-half-away-from-zero, clamping of full-scale and out-of-range values, and 960 / 2880 / 3840 B for 5 ms of stereo / 5.1 / 7.1. These need no audio hardware or FFmpeg. - UI (
settings-behavior.test.ts):high_quality_audiois visible only forac3/eac3. - Tests were not run as part of writing this description.
User Stories
Coverage of the Vibepollo user stories from draoncc/Vibepollo-Old#11, checked against this diff. Client-side and Apollo stories live in other repositories and are out of scope for this PR.
| # | Story | In this diff |
|---|---|---|
| 1 | PCM is a choice of the audio format | ✅ config.cpp, settingsSchema.ts |
| 2 | Audio format still defaults to Opus | ✅ default unchanged |
| 3 | Docs state PCM bandwidth per layout | ✅ configuration.md, en.json |
| 4 | Docs say PCM supports 7.1, unlike Dolby audio | ✅ configuration.md |
| 5 | PCM appears in the web UI dropdown | ✅ settingsSchema.ts, en.json |
| 6 | Clients that don't accept PCM get Opus, no error | ✅ configure_codec, tested |
| 7 | Log line when the client didn't accept PCM | ⚠️ not in this diff |
| 8 | Dolby audio behaves as today | ✅ existing Dolby rows keep passing |
| 9 | Video bitrate lowered for PCM, capped at 20% | ✅ rtsp.cpp, tested |
| 10 | PyroWave counts PCM + recovery in full | ✅ via existing branch, tested |
| 11 | PyroWave refuses a PCM session that can't fit | ✅ tested (400 at 9000 kbps) |
| 12 | PCM packets encrypted like Opus/Dolby | ✅ encryption path untouched; no PCM-specific test |
| 13 | ANNOUNCE reply reports PCM | ✅ codec_name |
| 14 | Host fixes PCM packets at 5 ms | ✅ PCM_PACKET_DURATION, tested |
| 15 | Opus quality flag, surround params, high quality setting have no effect | ✅ configure_codec, tested |
| 16 | WebRTC audio unaffected | ✅ PCM path forwards samples under the same condition as the Opus and Dolby paths |
| 34 | PCM covered by the ANNOUNCE composition test | ✅ |
| 35 | Offer choice covered by tests | ✅ codec_offer |
| 36 | Sample conversion tested without hardware/FFmpeg | ✅ test_audio.cpp |
| 37 | Dolby names generalized | ✅ |
| 38 | Change kept small against upstream | ✅ use_pcm flag beside use_ac3 / use_eac3 |
Stories 17–33 and 40 (Artemis, Apollo, moonlight-common-c) are not part of this PR.