#6438
API session lock registry grows without bound for completed sessions
Bug
The OpenAI-compatible API stores one asyncio.Lock per user supplied session_id in a regular dictionary. Completed sessions are never removed, so sending requests with many distinct session IDs grows the API process memory indefinitely.
Reproduction
- Start the API server.
- Send one successful request to
/v1/chat/completionsfor each of many uniquesession_idvalues. - Inspect the app's session lock registry: it retains one lock for every completed session.
Expected behavior
Locks should remain available while requests for a session are active or waiting, then become reclaimable after the session's requests finish.
Proposed fix
Store the locks in a weak-value registry, matching the existing AgentLoop session-lock lifecycle. Add a regression test that completes a request and verifies its lock is not retained.
Activity
0No activity yet.