Upgrade 0.22.0 to 1.0.2 (macOS 15.1.1): host VPN does not work within vzNAT VM (but slirp does)

[marcelbeumer]

Description

Description

Host macOS 15.1.1 (24B91) with VPN from corporate that uses HTTP proxy.

With 0.22.0 vzNAT worked fine in combination with VPN+proxy.

default networking works:

limactl start \
  --name=proxytestok \
  --cpus=4 \
  --memory=8 \
  --vm-type=vz \
  --rosetta \
  --mount-type=virtiofs \
  --mount-writable
  template://ubuntu
  
.... inside VM

curl -x <proxy> http://www.google.com -v
(works)

vzNAT does not work:

limactl start \
  --name=proxytestbad \
  --cpus=4 \
  --memory=8 \
  --vm-type=vz \
  --rosetta \
  --mount-type=virtiofs \
  --mount-writable \
  --network=vzNAT \
  template://ubuntu

... inside VM

curl -x <proxy> http://www.google.com -v

*   Trying <proxyip>:8080...
* Connected to <proxyip> (<proxyip>) port 8080
> GET http://www.google.com/ HTTP/1.1
> Host: www.google.com
> User-Agent: curl/8.9.1
> Accept: */*
> Proxy-Connection: Keep-Alive
> 
* Request completely sent off

(hangs)

[nirs]

@marcelbeumer One of the changes since 0.22 preferring the additional network over the builtin user network, you can try this.

You probably see something like this in ip r:

$ ip r
default via 192.168.106.1 dev lima0 proto dhcp src 192.168.106.2 metric 100 
default via 192.168.5.2 dev eth0 proto dhcp src 192.168.5.15 metric 200 
...

To have the same setup of 0.22, you can change metric: 100 for the vzNAT interface:

% limactl edit /proxytestok
...
networks:
- vzNAT: true
  metric: 200

This will bring back the issues in 0.22 when install k8s when having 2 default routes with the same default metric, but maybe these issues are not affecting what you do.

Start the vm, you will see now:

$ ip r
default via 192.168.5.2 dev eth0 proto dhcp src 192.168.5.15 metric 200 
default via 192.168.106.1 dev lima0 proto dhcp src 192.168.106.2 metric 200 
...

Does it change anything?

[marcelbeumer]

Yes, with metric: 200 it works again! Output of ip r was similar to what you described, before and after.

Thanks a lot. What should we do with this GH issue?

[nirs]

Great that it works, let keep the issue open for now. If this issue is very common maybe we need to change the default metric to keep the old behavior.

[nirs]

@jandubois do you think we should change the metric to the default value to keep 0.22 behavior? We assumed that adding additional network means you want to use it, so it should be preferred. But it seems that some features work only with the user network, so maybe lima is not ready for preferring the additional network.

For the use case of multiple k8s clusters using shared network using custom metric works fine, so using default metric is fine.

We have only one report so far, so I'm not sure it is worth a change. And we also need to understand why this case is solved by using non-optimal configuration of 2 default routes with same metric.

[jandubois]

I'm not keen of changing behaviour, and then changing it back, unless it is absolutely necessary.

It would be better to understand first why this is breaking some scenarios, and to figure out if there is a different way to make them work again automatically.

[nirs]

Host macOS 15.1.1 (24B91) with VPN from corporate that uses HTTP proxy.

@marcelbeumer is it possible to reproduce this on another machine without your corporate VPN/HTTP proxy?

If you and find how to reproduce it, we can try to understand why it does not work with the default confguration.

Also maybe you try add a note in the relevant docs about this issue with the solution?

[auxdsimon]

I am on lima 1.1.1 and having a VM created with 0.2x before. It worked before, but probably I got an update and even upgraded the lima and got broken with my company OPENVPN setup. I edited the lima.yml and added the metric: 200 and it solved the problem.