fix(sse): streamed tool_calls arguments are duplicated when credentialRedactionEnabled=true

OmniRoute Version

3.8.52 — published image diegosouzapw/omniroute:3.8.52 (digest sha256:b08a12e8…, built 2026-10-20T01:40Z, package.json → 3.8.52)

Installation Method

Docker / Docker Compose

Operating System

Linux

OS Version

Ubuntu 24.04 (kernel 6.8.x), single container, SQLite storage, Redis disabled

Node.js Version

v26.12.0 (as shipped inside the image)

Provider(s) Involved

ollama-cloud. The defect sits in the SSE re-emission layer, so it should not be provider-specific — but this is what was measured.

Model(s) Involved

deepseek-v4.1-flash (priority combo deepseek-v4.1-flash; first leg ollama-cloud/deepseek-v4.1-flash:1017)

Client Tool

Hermes Agent — OpenAI Python SDK, stream: true, stream_options.include_usage: true, 42 tool definitions.

Description

When settings.credentialRedactionEnabled = true, streamed tool calls arrive with duplicated argument fragments. A single tool call whose arguments are ~1.5–3 KB is emitted as three deltas:

3.8.51  → delta sizes [1676]              arguments concat = valid JSON
3.8.52  → delta sizes [1524, 1524, 512]   first fragment repeated verbatim

Clients that concatenate choices[].delta.tool_calls[].function.arguments (the OpenAI SDK, therefore every SDK-based agent) end up with prefix + prefix + tail, i.e. invalid JSON, and refuse to execute the tool:

⚠️  Truncated tool call detected — retrying API call (2/4)… (4/4)
⚠️  Truncated tool call response detected again — refusing to execute incomplete tool arguments.

This makes 3.8.52 unusable for streamed tool-calling agents whenever credential redaction is enabled, while the gateway itself reports success: call_logs stores finish_reason: "tool_calls" with complete, valid arguments, HTTP 200, normal latency, no cooldown/queueing (added_wait_ms = 0).

Steps to Reproduce

1. Set credentialRedactionEnabled = true (Settings → Guardrails) — note the code default is false, so this only affects opt-in installs.
2. Send a streaming chat completion with at least one tool and force a tool call whose arguments exceed ~512 characters (a terminal/write_file-style tool with a multi-KB script works well):

   curl -sN -H "Authorization: Bearer $KEY" -H 'content-type: application/json' \
     -X POST http://127.0.0.1:20128/v1/chat/completions --data-binary @request.json

3. Concatenate delta.tool_calls[].function.arguments per (choice index, tool index) and JSON.parse() the result.
4. Repeat with credentialRedactionEnabled = false, and/or against 3.8.51 — both are clean.

Expected Behavior

Each argument fragment is delivered exactly once, and the concatenation parses as JSON — i.e. the same bytes the gateway itself reassembles and records in call_logs.

Actual Behavior

The first buffered fragment is emitted twice, followed by a fixed 512-character tail. Observed on 11/11 streamed tool calls (3 sequential + 8 concurrent, same request body):

Content (delta.content) is unaffected (10 484 chunks, no systematic duplication), and small tool calls (≤ ~512 chars, single delta) never reproduce — which is why quick smoke tests pass.

Test Impact

No existing test asserts “concatenated streamed tool-call arguments equal the completed tool-call arguments”, so this regression ships green. Suggested guard: a unit test on drainToolCalls (emit-only-the-new-tail invariant) plus an e2e that streams a >1 KB tool call with credentialRedactionEnabled=true and round-trips the concatenation through JSON.parse.

Error Logs / Output

Client side (OpenAI-SDK based agent):

⚠️  Truncated tool call detected — retrying API call (2/4)…
⚠️  Truncated tool call detected — retrying API call (4/4)…
⚠️  Truncated tool call response detected again — refusing to execute incomplete tool arguments.
⚠️  Response truncated (finish_reason='length') - model hit max output tokens

Gateway side for the very same request (HTTP 200, 4.1 s, completion_tokens: 859):

{"choices":[{"message":{"role":"assistant","content":"","tool_calls":[{"function":{"name":"terminal","arguments":"{\"command\":\"set -u …\"}"}}]},"finish_reason":"tool_calls"}]}

Screenshots

Not applicable — SSE byte capture is in the table above.

Additional Context

Looks like it comes from the new streaming credential transform, which does not exist at the v3.8.51 tag:

src/lib/streamingCredentialTransform.ts

const DEFAULT_WINDOW_SIZE = 512;           // ← matches the fixed 512-char tail

function drainToolCalls(delta, choiceBuf, toolEntries) {
  …
  if (choiceBuf.toolArgs) {
    // pushes the WHOLE accumulated buffer, not just the newly flushed tail
    toolCalls.push({ index: 0, function: { arguments: choiceBuf.toolArgs } });
    choiceBuf.toolArgs = "";
  }

Gate: isCredentialTransformEnabled() returns true when credentialRedactionEnabled === true and the credential-masker guardrail is not disabled — which is exactly our configuration (we opted in to secret masking).

main (v3.8.53 line, HEAD edab7c3a) still contains the same code, so the next release would keep the behaviour unless fixed.

Workaround that we verified end-to-end (fixes it immediately, no restart of anything else): set credentialRedactionEnabled = false → delta sizes collapse back to a single fragment, arguments valid, three consecutive streamed tool calls all clean. Rolling the image back to 3.8.51 also works (its additive DB migration is rollback-safe, and 3.8.51 still authenticates the enc:v1 API-key rows by hash).

Validation Plan

1. Repro before/after with the byte-level delta assertion above (sequential + 8-way concurrent).
2. Add the e2e described under Test Impact and run it with credentialRedactionEnabled toggled both ways.
3. Sanity-check the redaction path still works: a payload containing a well-known API-key pattern must be masked exactly once, and the streamed tool-call arguments must still parse after masking.