#5849
Issues/HKUDS/nanobot/#5849

Auto-compaction deadlock: summarize_transcript has no token-budget guard, compaction can never recover once history exceeds input budget

OpenKrislu1221opened 1mo ago

Summary

The automatic context-compaction path (summarize_transcript) sends the entire session history + system prompt to the summarization model without any token-budget protection. The manual path (archive_session) correctly uses get_history(max_tokens=budget) chunking, but the automatic path does not.

Consequence

Once a session's history exceeds the model's input budget, a deadlock occurs:

  1. Governor blocks the request because input > budget ("Unable to compact context")
  2. Compaction itself requires an LLM call whose input (full history + prompt) also exceeds the budget
  3. That call is also blocked / falls back to raw archiving, so history never shrinks
  4. The session is permanently stuck — the only recovery is /new or manually clearing history

In our case the trigger was a misconfigured contextWindowTokens (64000 vs the model's real 1M window), which made fixed overhead (~92k tokens of system prompt + tool definitions) exceed the local input budget (~54784). But even with a correctly sized window, any session whose history grows past the window hits the same permanent deadlock — a bigger window only delays it.

Observed behavior

Compressing context… → Unable to compact context (input 91985 tokens > budget 54784)

Repeated on every turn; context never reduced.

Suggested fix

Apply the same budget guard used by archive_session to summarize_transcript: when the history passed to the summarizer exceeds the model's input budget, truncate/chunk it via get_history(max_tokens=budget - reserved_for_prompt) (or fall back to keeping the most recent N messages) so compaction can always make progress.

Environment

  • nanobot v0.3.5 (upgrade that made the failure visible: precise tiktoken estimation + strict governor)
  • macOS, GLM-5.3-Flash endpoint

Activity

1
chengyongru1mo ago

I tested this against current origin/main (12a9a8a692c7e8fc6d3d293afdddcdaa390436df) using the real Zhipu GLM-5.3-Flash API and a temporary copy of a local session. I could reproduce the oversized archive-input condition, but I could not reproduce the permanent compaction deadlock described in this issue.

Test details:

  • The source session contained 7,127 messages (21.1 MB) and already had last_archived=7126. Running it as-is did not trigger compaction. The turn completed successfully with 10,781 provider input tokens.
  • In that real workspace, the base system prompt was approximately 5.0-5.6k tokens. The 22 built-in tool schemas added approximately 5.35k tokens. There was no 50k-token system prompt.
  • To exercise the reported failure path, I reset the archive markers only in the temporary session copy, making all 7,127 messages replayable. The resulting archive request was estimated at 6,220,284 tokens.
  • With context_window_tokens=200000, the archive input budget was 191,808 tokens. Current main detected that the archive request did not fit and used the raw-checkpoint fallback. The subsequent GLM request used 14,740 input tokens and completed successfully.
  • I repeated the same replay test with context_window_tokens=64000. The regular input budget was 54,784 tokens (64000 - 8192 max output - 1024 safety buffer), while the archive guard used 55,808 tokens (64000 - 8192). Current main again used the raw-checkpoint fallback, then sent a 14,740-token request to GLM, which completed successfully.

The original config and session were not modified. One limitation is that the configured XMind MCP server was not connected during the probe, so its tool schemas were not included in the prompt measurements.

My conclusion from these runs is:

  • The condition "archive input exceeds the configured budget" is reproducible.
  • The reported loop (Compressing context -> Unable to compact context on every turn) is not reproduced on current main. Current main preserves liveness by falling back to a raw checkpoint.
  • In this oversized-history case, current main does not produce an LLM-generated semantic summary. A bounded-history change may improve compaction quality, but these results do not show that it fixes a current-main permanent deadlock.
  • The 54,784 value in the 64k setup is a local input budget, not the GLM model's context limit or the system-prompt size.

Reproducing the original hard deadlock may require the exact v0.3.5 revision and the reporter's session/tool state.