What happened?
bedrock-converse-stream.ts does new BedrockRuntimeClient(config) inside stream(), so every model call gets a fresh client: new credential provider and new HTTP/2 session.
With an SSO profile that means every call does a GetRoleCredentials round trip to portal.sso plus a new TLS handshake to bedrock-runtime. I measured 2 new TLS handshakes per call, around 0.35 s before the request even starts (0.24 s SSO + 0.11 s TLS from eu-west-1). In an agent loop with ~35 calls per run that adds up fast. The old SSO sockets also stay open, I had 29 idle connections to portal.sso in one process.
I patched it locally to keep one client per config (region, profile, endpoint, hashed credentials, proxy) in a small LRU with an idle timeout, and dropping the cached clients on credential/auth errors so a refreshed or rotated key is picked up. Time to first byte on EU Haiku went from ~1.25 s to ~0.69 s, and calls after the first one do 0 handshakes.
Steps to reproduce
- Use
amazon-bedrockwith an SSO profile (AWS_PROFILE). - Run any session with a few tool calls.
- Watch
lsof -i(or count TLS handshakes): every call opens a new connection toportal.sso.<region>.amazonaws.comandbedrock-runtime.<region>.amazonaws.com.
Expected behavior
The client, credential provider and HTTP/2 connection are reused across requests with the same config.
Version
1.4.0, still the same on main (cc9c52310)
Happy to open a PR for this if you want it.
Activity
0No activity yet.