#620

win-sshproxy: named pipe connection not closed after SSH channel EOF causes non-Go API clients to hang

Openli-zhixinopened 7mo ago

Problem

When win-sshproxy forwards traffic between a Windows named pipe (\.\pipe\docker_engine) and an SSH tunnel to Podman's Unix socket, the named pipe connection is not properly closed after the remote SSH side sends EOF. This causes non-Go API clients (Node.js, .NET, etc.) to hang indefinitely when using Podman's exec or attach APIs on Windows.

Root Cause Analysis

In pkg/sshclient/ssh_forwarder.go, the forward() function handles bidirectional data relay:

func forward(src io.ReadCloser, dest CloseWriteStream, complete *sync.WaitGroup) {
    defer complete.Done()
    _, _ = io.Copy(dest, src)
    _ = dest.CloseWrite()   // ← Problem is here
}

When the SSH channel sends EOF (e.g., Podman closes a hijacked exec connection):

  1. io.Copy(pipe, ssh) returns
  2. pipe.CloseWrite() is called
  3. go-winio's CloseWrite() implementation writes a zero-byte message to the named pipe to signal EOF
  4. Non-Go clients don't interpret zero-byte messages as EOF — Node.js (libuv), .NET NamedPipeClientStream, and other clients simply ignore it
  5. The pipe connection stays open, the client never receives an end/close event
  6. Both forward() goroutines never complete (the pipe→ssh direction blocks on read), so src.Close() and dest.Close() in the cleanup goroutine are never called either

Impact

This causes connection leaks that eventually exhaust all named pipe instances, making Podman completely unresponsive on Windows. This is the root cause of containers/podman#24798.

Affected users: anyone using Podman on Windows with non-Go API clients, including:

  • testcontainers-node (Node.js) — container.exec() hangs forever
  • testcontainers-dotnet (.NET) — NamedPipeClientStream hangs
  • Docker SDK clients — any client connecting via \.\pipe\docker_engine

Reproduction

Environment: Podman 5.7.1, Windows 11, WSL2, win-sshproxy via npipe:////./pipe/docker_engine

# 1. Start a container
podman run -d --name exec-test alpine sleep 300

# 2. Create exec instance via named pipe (works fine)
node -e "
const http = require('http');
const body = JSON.stringify({Cmd:['/bin/sh','-c','echo hello'],AttachStdout:true,AttachStderr:true});
http.request({socketPath:'//./pipe/docker_engine',path:'/v1.41/containers/exec-test/exec',method:'POST',
  headers:{'Content-Type':'application/json'}}, res => {
  let d=''; res.on('data',c=>d+=c); res.on('end',()=>console.log(res.statusCode,d));
}).end(body);
"
# Output: 201 {"Id":"<EXEC_ID>"}

# 3. Start exec via named pipe — stream never closes
node -e "
const http = require('http');
const body = JSON.stringify({Detach:false,Tty:false});
http.request({socketPath:'//./pipe/docker_engine',path:'/v1.41/exec/<EXEC_ID>/start',method:'POST',
  headers:{'Content-Type':'application/json'}}, res => {
  console.log('status:', res.statusCode);
  res.on('data', c => console.log('data:', JSON.stringify(c.toString())));
  res.on('end', () => console.log('END — stream closed'));
}).setTimeout(10000, function() { console.error('TIMEOUT — stream never closed'); this.destroy(); })
  .end(body);
"
# Output:
#   status: 200
#   data: "...hello\n"
#   TIMEOUT — stream never closed    ← BUG: 'end' event never fires

# 4. Same request via direct SSH tunnel — stream closes correctly
ssh -f -N -L 12345:/run/podman/podman.sock root@127.0.0.1 -p 60022 -i <identity_file> -o StrictHostKeyChecking=no
# (repeat exec create + start against http://127.0.0.1:12345 instead of named pipe)
# Output:
#   status: 200
#   data: "...hello\n"
#   END — stream closed              ← Works correctly, bypassing win-sshproxy

The comparison between step 3 (via win-sshproxy, hangs) and step 4 (via direct SSH tunnel, works) confirms the issue is in win-sshproxy's named pipe handling.

Suggested Fix

When the SSH channel sends EOF on a hijacked/exec connection, CloseWrite() alone is insufficient for non-Go clients. The full Close() should be called on the pipe connection to properly signal EOF to all client implementations.

One approach: detect when io.Copy returns due to EOF on the SSH side and immediately close the pipe connection, rather than relying on CloseWrite() which only works with Go's go-winio reader.

Related Issues

  • containers/podman#24798 — "Podman Becomes Unresponsive During Test Execution on Windows" (downstream symptom)
  • containers/podman#6853 — Original exec stream closure issue (fixed for direct API access, but not through win-sshproxy)

Activity

0

No activity yet.