Write ADR for load-balancers

[garloff]

As a KaaS operator, I want to understand exactly what standards my load-balancer must fulfill by reading the relevant ADR.

TODO: Write ADR (according to SCS-0001 standard) to document the requirements tested for in #212.

Definition of Ready:

• User Story is small enough to be finished within one sprint
• User Story is clear and understood by the whole team
• Acceptance criteria are defined
• Acceptance criteria are clear and understood by the whole team

Definition of Done:

• All acceptance criteria are met
• Changes have been reviewed
• CI tests have run successfully
• Documentation has been updated
• Release Notes have been updated

[garloff]

nginx-ingress expects that the single nginx instance on a multi-node cluster works despite externalTrafficPolicy: Local (which means kube-proxy is NOT forwarding connections) on a multi-node cluster. Apparently, nginx-ingress expects the LoadBalancer to detect which nodes are up and load-balance only to the ones where nginx is listening.
This is the most efficient way (by avoiding unneeded forwards), but one could question whether loadbalancers should always detect nodes being down. With the default setting of the OpenStack Cloud Controller Manager, loadbalancers do NOT fulfill this: They don't automatically get a health monitor. This is not unreasonable either -- health monitors can always be requested, though unfortunately, this is not standardized but can only be achieved by an OpenStack specific annotation. Which makes things non-portable :-(
So we will have to default to health-monitors be enabled ...

[garloff]

TODO:

• Investigate why the default to NOT have health monitors was chosen.
• Investigate whether opt-out (by specific annotation) is still possible
• Compare to other CCMs and their defaults

[garloff]

@garloff and @joshmue to look at OCCM in first week of Jan.

[garloff]

Needs more discussion:

• Making traffic flow work (with the help of the health monitor) may result in something half working for the user:
  • externalTrafficPolicy: Local results in working traffic
  • BUT the client IPs are not preserved!

So: Investigate whether we can address the client IP thing:

• proxy protocol (working for http/https like protocols at least)
• generic approach using L3 loadbalancer (DSR like), maybe possible with ovn magic ...

[jschoone]

Hi @joshmue. Are we still on track with this?

[joshmue]

The ADR in SovereignCloudStack/standards#169 is pretty much complete. IIRC, the decision that is documented in this record did also match the consensus in a (team?) meeting.
Hence, it just needs to be pushed forward according to the according process.

[joshmue]

Refs SovereignCloudStack/standards#648

Will assign @cah-hbaum, please feel free to re-assign me, if I can/should do anything here.