private gateway attach to physical network issue

[bradh352]

I've got these physical networks:

And I want to add a private gateway to access the mgmt network to allow access to few ports (ceph via the mgmt/hypervisor network in this case):

Before I attempt to add a private gateway, I check my network bridges:

ceph		8000.de95ff0e22c3	no		vxlan200
hypervisor		8000.0625c5991a54	no		vxlan100
public		8000.d286dc39fb05	no		vnet1
							vxlan2

I can see vnet1 for my VPC public ips attached to public which is right. But otherwise no vnets are attached to those bridges which is right as well.

So I create a private gateway in my VPC like this:

It creates fine, but when I look at bridge config, it attached to the wrong network!

ceph		8000.de95ff0e22c3	no		vxlan200
hypervisor		8000.0625c5991a54	no		vxlan100
public		8000.d286dc39fb05	no		vnet1
							vnet11
							vxlan2

I would have expected vnet11 to attach to hypervisor but instead it attached to public. Am I doing something wrong here? Or misunderstanding something? Or is this a bug?

I'm running 4.21.0

[bradh352]

I verified with cmk as well to make sure its not a UI bug. It definitely attaches it to a physical network other than what I specified even though the UUIDs all match.

Its like its ignoring the kvmnetworklabel on the physical network and just choosing the one with the default gateway.

(localcloud) 🐱 > createPrivateGateway gateway=10.10.100.99 ipaddress=10.10.100.99 netmask=255.255.255.0 vpcid=62940001-aa31-4da1-abf4-5a8c4f7a7f76 aclid=346df974-6f58-4a8e-94ab-a6559e7bbf2f physicalnetworkid=1b567de0-2a6d-4583-994d-c250b58b5e0d vlan=vlan://untagged
{
  "privategateway": {
    "account": "PrjAcct-infra-1",
    "aclid": "346df974-6f58-4a8e-94ab-a6559e7bbf2f",
    "aclname": "hypervisor",
    "domain": "ROOT",
    "domainid": "eea9edcb-8e65-11f0-8cb1-fa96e6c9c041",
    "domainpath": "/",
    "gateway": "10.10.100.99",
    "id": "a675c8c5-21ab-4ec5-bb74-050401c2b70c",
    "ipaddress": "10.10.100.99",
    "netmask": "255.255.255.0",
    "physicalnetworkid": "1b567de0-2a6d-4583-994d-c250b58b5e0d",
    "project": "infra",
    "projectid": "92825ef4-53b0-45b8-a814-35e292871d2a",
    "sourcenatsupported": false,
    "state": "Ready",
    "vlan": "vlan://untagged",
    "vpcid": "62940001-aa31-4da1-abf4-5a8c4f7a7f76",
    "vpcname": "infra_vpc",
    "zoneid": "841782a0-eb61-46f3-ba10-3386ed334df0",
    "zonename": "testlab"
  }
}

[bradh352]

Inspecting the database gave me a hint, the traffic type in the database said "Guest" and I didn't have any guest network defined for my 'mgmt' physical network. I'd think it should have errored out if it was needed, so I don't honestly know how it chose what to use.

I've pasted the instructions below I created for myself. That said, its still not perfect, if you have a deny rule in your network acls for egress, nothing works. What's even more frustrating is if you temporarily switch it to default_allow it works, then when you switch it back to your ACL, it still works! Restarting the VPC is the only way to know if it will really persist.

Access to Ceph Network

We need to be able to add a private gateway to access the hypervisor network which is also our management network.

In order to do that we must add a tag to our management network and then add support for guest traffic types and finally set the traffic label to match our interface name.

1. Under Infrastructure -> Zones -> Select Zone -> Physical Networks, choose the mgmt network.
2. Click the pencil button to update the physical network and add a tag. I just used 'mgmt', not sure the tag matters but it won't let us add multiple guest networks in the zone without them having different tags.
3. Click the [+] button to add a traffic type and choose guest, set the isolation method to vlan
4. Finally click the merge or branch looking button which is really update traffic labels then select the Guest network and set the kvm traffic label to the network interface name, in our case hypervisor

[weizhouapache]

@bradh352
I have created a github issue for tracking : #11798

closing this