vz: add SSH over AF_VSOCK #3979
Conversation
norio-nomura
force-pushed
the
ssh-over-vsock-on-vz
branch
from
146e87a to
6ba3d31
Compare
pkg/driver/vz/vsock_forwarder.go
Outdated
| if err != nil { | ||
| return err | ||
| } | ||
| logrus.Infof("started vsock forwarder: localhost:%d -> vsock:%d on VM", hostPort, vsockPort) |
There was a problem hiding this comment.
Can be another PR, but wondering if we can further optimize the performance with (an equivalent of) systemd-ssh-proxy — SSH client plugin for connecting to AF_VSOCK and AF_UNIX sockets
https://www.freedesktop.org/software/systemd/man/256/systemd-ssh-proxy.html
There was a problem hiding this comment.
I don't understand the meaning of the comment.
There was a problem hiding this comment.
I thought it might be possible to eliminate the TCP->vsock forwarder and let ssh directly connect to the vsock.
There was a problem hiding this comment.
VM's VSOCK should not be open to the host unless the process that is starting the VM is mediated. As this PR is doing.
norio-nomura
force-pushed
the
ssh-over-vsock-on-vz
branch
2 times, most recently
from
4ea3d0f to
a549d55
Compare
|
Ubuntu 24.04 and earlier, SSH over AF_VSOCK can be enabled with: provision:
- mode: yq
path: /etc/systemd/system/ssh.socket.d/vsock.conf
format: ini
expression: |
.Socket.ListenStream="vsock::22"and upgraded SSH tracked at sshd socket activation does not support AF_VSOCK |
How to enable SSH over AF_VSOCK on Ubuntu 20.04, 22.04, and 24.04Ubuntu 20.04, 22.04:Since the patched version of SSH has already been released, it requires:
$ limactl start template://ubuntu-20.04 --rosetta --containerd=none --set '.provision|=.//empty + [{
"mode": "yq",
"format": "ini",
"path": "/etc/systemd/system/ssh.socket.d/vsock.conf",
"expression": ".Socket.ListenStream=\"vsock::22\""
}, {
"mode": "system",
"script": ("#!/bin/bash
set -eux -o pipefail
systemctl is-enabled ssh.service || exit 0
# use socket based activation
systemctl disable --now ssh.service
systemctl enable --now ssh.socket
"|. style="literal")
}]'
$ limactl restart ubuntu-20.04 2>&1 |grep -i vsock
time="2025-09-07T17:01:40+09:00" level=info msg="[hostagent] started vsock forwarder: localhost:59291 -> vsock:22 on VM"
time="2025-09-07T17:01:40+09:00" level=info msg="[hostagent] Detected SSH server is listening on the vsock port; changed localhost:59291 to proxy for the vsock port"Ubuntu 24.04:
$ limactl start template://ubuntu-24.04 --rosetta --containerd=none --set '.provision|=.//empty + [{
"mode": "yq",
"format": "ini",
"path": "/etc/systemd/system/ssh.socket.d/vsock.conf",
"expression": ".Socket.ListenStream=\"vsock::22\""
}, {
"mode": "system",
"script": ("#!/bin/bash
ss -l --vsock|grep \*:22 -q && exit 0
apt-get --update install --assume-yes openssh-server
"|. style="literal")
}]'
$ limactl restart ubuntu-24.04 2>&1 |grep -i vsock
time="2025-09-07T17:01:56+09:00" level=info msg="[hostagent] started vsock forwarder: localhost:59299 -> vsock:22 on VM"
time="2025-09-07T17:01:56+09:00" level=info msg="[hostagent] Detected SSH server is listening on the vsock port; changed localhost:59299 to proxy for the vsock port"Edit: The patched version of openssh-server has been released on Ubuntu 24.04 |
norio-nomura
force-pushed
the
ssh-over-vsock-on-vz
branch
2 times, most recently
from
2d98aa5 to
5161063
Compare
|
I'm considering another PR for the port forwarder implementation to VSOCK, which can be defined in |
norio-nomura
force-pushed
the
ssh-over-vsock-on-vz
branch
2 times, most recently
from
e3bc0bd to
9238459
Compare
The patched version of SSH has been released (2025/09/09), update openssh-server to latest release. |
|
To support custom |
Since systemd v256 (Ubuntu 24.10), SSH is bound to AF_VSOCK port 22. https://github.com/systemd/systemd/releases/tag/v256 > - If the system is run in a VM providing AF_VSOCK support, it automatically binds sshd to AF_VSOCK port 22. https://discourse.ubuntu.com/t/oracular-oriole-release-notes/44878 > - When sshd is installed on a system, a new systemd generator, systemd-ssh-generator binds a socket-activated SSH server to local AF_VSOCK and AF_UNIX sockets under certain conditions. This changes to delay starting SSH port forwarding until the SSH server on the VM becomes ready. If AF_VSOCK port 22 can be connected, start a local SSH port as a proxy for AF_VSOCK port 22, instead of starting gvisor's port forwarder. SSH over VSOCK is faster than SSH over gvisor's port forwarder. This change is opt-out because it requires VZ and VM with systemd v256+, setting `LIMA_SSH_OVER_VSOCK=true` does not mean it works. To disable, set `LIMA_SSH_OVER_VSOCK=false`. Signed-off-by: Norio Nomura <[email protected]>
norio-nomura
force-pushed
the
ssh-over-vsock-on-vz
branch
from
9238459 to
8119bb9
Compare
done. |
This added an entry point |
Description:
Since systemd v256 (Ubuntu 24.10), SSH is bound to AF_VSOCK port 22.
https://github.com/systemd/systemd/releases/tag/v256
https://discourse.ubuntu.com/t/oracular-oriole-release-notes/44878
This PR changes to delay starting SSH port forwarding until the SSH server on the VM becomes ready. If AF_VSOCK port 22 can be connected, start a local SSH port as a proxy for AF_VSOCK port 22, instead of starting gvisor's port forwarder.
SSH over VSOCK is faster than SSH over gvisor's port forwarder.
This change is opt-out because it requires VZ and VM with systemd v256+,
setting
LIMA_SSH_OVER_VSOCK=truedoes not mean it works.To disable, set
LIMA_SSH_OVER_VSOCK=false.Benchmark logs:
On MacBook Pro 14 inch, 2023 with Apple M2 Pro
SETUP:
GRPC Port Forwarder (Current):
SSH Port Forwarder on gvisor's virtual network (Old):
SSH Port Forwarder over AF_VSOCK (New):