#6438

API session lock registry grows without bound for completed sessions

OpenSC-10749opened 6d ago

Bug

The OpenAI-compatible API stores one asyncio.Lock per user supplied session_id in a regular dictionary. Completed sessions are never removed, so sending requests with many distinct session IDs grows the API process memory indefinitely.

Reproduction

  1. Start the API server.
  2. Send one successful request to /v1/chat/completions for each of many unique session_id values.
  3. Inspect the app's session lock registry: it retains one lock for every completed session.

Expected behavior

Locks should remain available while requests for a session are active or waiting, then become reclaimable after the session's requests finish.

Proposed fix

Store the locks in a weak-value registry, matching the existing AgentLoop session-lock lifecycle. Add a regression test that completes a request and verifies its lock is not retained.

Activity

0

No activity yet.