Feature/Improvement: pick LoadBalancer Strategy based on annotation

[vrann]

When working on the new mechanism of processing of the activations we haven't found a better extension point on where it could be plugged into the system other than changing the logic of Load Balancer.

We don't want to introduce backward incompatibilities of any kind, so this approach must work alongside the default one. This motivates us to introduce a mechanism that would allow selecting the Load Balancers based on some characteristics of activations. We considered kind as one possibility for such characteristic, and it could be added later. For now, annotation looks like the most straightforward way (with fewer drawbacks)

What we propose is to introduce is a new load balancer called MuxBalancer. When action is created, a developer can specify "activationStrategy" annotation. If OpenWhisk is configured to use MuxBalancer, it will pick up the annotation value and will look in the map of the configured activation strategies for that alias. In case if the alias is found, it will delegate the publishing of the activation to the LoadBalancer corresponding to the alias. Otherwise, MuxBalancer will use the "default" Load Balancer to delegate the publication.

The configuration of the MuxBalancer looks like folloving:

whisk.loadbalancer {
    strategy {
      default = "fully.qualified.class.name.for. LoadBalancer0"
      custom = {
           "activationStrategyName1" = "fully.qualified.class.name.for. LoadBalancer1"
           "activationStrategyName2" = "fully.qualified.class.name.for. LoadBalancer2"
            ...
      }
    }
}

A developer would use it in the following way:
wsk action create hello hello.js --annotation activationStrategy activationStrategyName1

Potentially we could add the possibility to restrict kinds on which load balancer would apply, etc

[bdoyle0182]

I'll just link this comment here from the PR:

#5091 (comment)

But will just TL;DR I don't think load balancing along action lines will work with the current architecture until the new scheduler is released

[rabbah]

If I am following the loadbalancer change proposed, it should work with both the existing and etcd schedulers - for the existing algorithm, there is a new dispatcher which sends activations either through the "classic" path (e.g., sharding load balancer), or the new annotation-based scheduler. For the etcd scheduler, it could work in a similar way but might actually also end up being simpler.

[bdoyle0182]

Maybe I'm misunderstanding what the proposed change is. Wouldn't the existing sharding load balancer need to have knowledge of all activations that are sent to an invoker to properly balance? If you have a new method to load balance and both methods share the same invokers, the sharding load balancer needs to be aware of the activations sent on the new method otherwise it may over-provision an invoker right?

It makes sense if you have two invoker pools that use different load balancing strategies, but that kind of defeats the purpose of running two strategies at once across action lines.

[vrann]

@bdoyle0182 The initial intent of this work is to make it possible to have a pool of containers that are not managed by traditional lb->kafka->invoker path, and instead, the controller would be able to place calls to those containers directly.

The original motivation for this is to switch the approach from having the container which is bound to the action and instead to have the container (or pool of containers) bound to a tenant. We will be talking more about this approach soon in a separate issue/PR.

This PR enables a way to come up with other activation strategies of such kind later.

It is possible that there will be two container pools, one with an invoker (managed by sharding lb) and one without. Or just one, and the invoker would be completely disabled.