VertX Gateway concurrent blocking calls

[lesinjan]

Hi!

We are trying to use Apiman Vert.X GW (v2.0.0 1 http verticle) in our solution and we have run into performance problems for the following scenario:

Client1 is calling our backend service through Apiman Vert.X GW.
Backend receives this request, does it's own stuff and notifies Client2, that he has to supply response.
Client2 calls our backend through the same Apiman Vert.X GW.
Backend matches response with waiting request and send response to Client1.

Picture of this scenario below:

Obviously this makes Client1 call blocking and in the worst case scenario there is no guarantee Client2 will ever respond and request will wait till timeout is reached.

Problem happens if there are multiple concurrent request from Client1. As soon as the concurrency reaches 5 no more calls pass through to our backend system. What is worse, Client2 responses does not reach backend as well, therefor Client1 calls keep on hanging on till our backend gives up and generates 408 as response at which point connection is freed up and next call reaches backend.

Looking at Apiman Vert.X GW code it seems, that problem happens because only one HttpClient is created for all the calls all the HttpConnectionOptions being equal. More so Http max pool size is left at its default (5). Which as far as I can tell results in only 5 concurrent calls per HttpClient going through.

Now, I can't help to think, that I am doing something wrong or misinterpreting code. It really seems strange, that performance oriented GW would come with such bottleneck. Any ideas?

PS Right now we have circumvented the problem by spinning up multiple http verticles, but this does not feel right as the whole point of migrating to Vert.X GW was to avoid excessive threading. Adding config param to specify max http connection pool size and creating separate httpclient for each client app, so that different clients don't fight over the same connections would seem to be a better approach.

Regards,
Janis

[msavy]

I'll need to find some time to figure out the nicest approach to solve this, as it's specific-ish to your kind of setup (first person to complain of this issue). I think your setup valid, so we should support that.

Just so you can try it out, I'll cut you a branch and let you set your own Vert.x client options. Consider the configuration keys to be entirely placeholder for now, please. I'm sure you can compile it yourself.

It's a rough hack so you can test it immediately.

https://github.com/msavy/apiman/tree/allow-custom-client-options

In short you can change any Vert.x client option you want.

  "connector-factory": {
    "class": "io.apiman.gateway.platforms.vertx3.connector.ConnectorFactory",
    "config": {
      // Your usual config here
       "httpClient": {
         "maxPoolSize": 100
       }
      }
    }
  }

Can discuss the other options later (new client each time, see if we can detect it's getting near pool size limit, etc).

[iamakashk]

Hi @lesinjan
Glad to hear that. 😃
Is it possible for you to share your configuration file for reference? I am still stuck on this issue. If you could share your email or twitter handle where we can connect ( if that is okay for you ).

[msavy]

Thanks for testing that @lesinjan.

It won't make it into this release (2.1 should be out in the next couple of days), but will certainly be in the next one (which should follow hot on its heels). 2.1.1 or something.

[iamakashk]

Hi @msavy

I am using apiman vertx gateway ( 1.5.7) and facing similar issue which @lesinjan was facing. Any suggestions on this to resolve this in 1.5.7 ?

[msavy]

I'm not currently planning to backport to 1.x.

I don't have the bandwidth, I'm afraid.

Your issue appears to be of a different source if the stack trace you posted is anything to go by. It is because something is blocking one of Vert.x's event loop threads. Possibly a custom policy?

[lesinjan]

Thanks @msavy !

I'm looking forward to seeing this improvement in one of the next releases. And I'm really impressed with the both prompt response and quick fix! Thanks!