Some VMware environments (Azure AVS for example) don't provide full VMware access and effectively prevent NBD access. We need to look at our options in this scenario and implement a fallback method.
There are a few things that we need to check:
- Can we easily detect such an environment or do we need the user to configure it on the source?
- What does access to the CBD data look like when using VMware HotAdd?
- Can we easily re-export NBD ourselves from a VMware VM with that CBD data?
In any case, we'd be looking at spawning a VMware VM per VM that we're migrating.
This will need a bit of logic to be able to create VMware VMs, place that VM on the exact same server as the VM we're migrating. Having some configuration in place to control what datastore and network to use.
The most likely path forward after that will be to flip the worker behavior to having that source VM act as the migration worker, having it get the credentials needed to access the Incus cluster and then use Incus' NBD endpoint to push the data rather than pull it.
But the main unknown at this point is how CBD/dirty bitmap tracking works so we can implement background disk transfer in a way similar to what we have today.
The main downside of this approach is that we now need to create potentially quite a lot of additional VMs on the source side. That's obviously far less ideal than just running the target VM and having it pull the data from the source.
So I don't think we should remove our current approach, it remains preferred. But introducing the logic described in this issue will make it potentially easier to support a variety of other sources by using a similar approach.
Some VMware environments (Azure AVS for example) don't provide full VMware access and effectively prevent NBD access. We need to look at our options in this scenario and implement a fallback method.
There are a few things that we need to check:
In any case, we'd be looking at spawning a VMware VM per VM that we're migrating.
This will need a bit of logic to be able to create VMware VMs, place that VM on the exact same server as the VM we're migrating. Having some configuration in place to control what datastore and network to use.
The most likely path forward after that will be to flip the worker behavior to having that source VM act as the migration worker, having it get the credentials needed to access the Incus cluster and then use Incus' NBD endpoint to push the data rather than pull it.
But the main unknown at this point is how CBD/dirty bitmap tracking works so we can implement background disk transfer in a way similar to what we have today.
The main downside of this approach is that we now need to create potentially quite a lot of additional VMs on the source side. That's obviously far less ideal than just running the target VM and having it pull the data from the source.
So I don't think we should remove our current approach, it remains preferred. But introducing the logic described in this issue will make it potentially easier to support a variety of other sources by using a similar approach.