Related: #17292 (same withEarlyStreamKeepalive catch-branch family, opposite symptom - we get a hard 502 instead of a silent clean EOF).
Symptom
Requests routed through ANY combo fail at ~55-86s with error: terminated -> HTTP 502 server_error after stream output. The SAME request body sent directly to the same model on the same connection completes fine, even at 290K tokens.
Three-way comparison (same request body, same provider key)
| Path | prompt_tokens | Result | Duration |
|---|---|---|---|
| Direct to upstream (curl) | 293,856 | 200, complete [DONE] | 20.5s |
| Gateway, single model (no combo) | 291,515 | 200, complete [DONE] | 115.3s |
| Gateway, combo (3 models) | 232,990 | 502 terminated | 55.8s |
| Gateway, combo (20 models) | 232,393 | 502 terminated | 75.3s |
| Gateway, combo | 223,644 | 502 terminated | 85.8s |
Threshold on the combo path: 42K-165,626 all succeed (59 requests, 3 failures - all above 220K); 223K/232K/252K fail every time.
Error signature
[STREAM] ...:DeepSeek-V4.1-Flash | 55736ms | error: terminated Server error for ... - 502 server_error after stream output (no model lockout, connection stays active for sibling models)
reasoning is always null on failures, out only 2,200-2,700 tokens. Upstream billing shows those same requests as fully successful (charged 115,014 in / 5,541 out), so upstream completed generation.
Ruled out
- Upstream window: provider says 1M; direct 293,856-token call returns 200 in 20.5s
- Proxy: proxy_enabled=0, reproduces with proxy off
- Version: identical on 3.8.51 and 3.8.52
- Network: 10/10 probes OK
- Daily quota: 24.5M / 45M (54.6%), previous window 41.2M (91.7%) - never hit
- context_cache_protection=0: pin still fires (driven by session_model_history, not this flag); clearing that table did not help
- reasoningTokenBufferEnabled=false: confirmed [rc-backfill] dropped to 0 lines; still fails
- Combo size: 3-model combo fails identically to 20-model
Repro
- openai-compatible-chat connection with operator baseUrl
- Priority combo containing that model (3 models is enough)
- Send a chat completion through the COMBO with >220K input tokens, stream:true, reasoning_effort:high, ~24 tool definitions
- Stream starts, reasoning emitted, answer body begins, then connection drops at 55-86s -> 502
- Same request with model=/ -> completes normally
Expected
A combo request should complete anything the underlying provider can serve directly. The pinned-model branch logs "routing directly to pinned context model" yet still fails while a genuine direct dispatch succeeds.
Full report with logs available on request.
Activity
0No activity yet.