Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rocket dropping connections on high cpu usage #1254

Closed
xylix opened this issue Mar 7, 2020 · 2 comments
Closed

Rocket dropping connections on high cpu usage #1254

xylix opened this issue Mar 7, 2020 · 2 comments
Labels
duplicate This issue or pull request already exists

Comments

@xylix
Copy link

xylix commented Mar 7, 2020

Bug Reports

  1. Rocket 0.4.3

  2. Fresh Arch linux (updated as of 7.3.2020) VM, Ubuntu 18.04 (both in a virtualbox VM and on travis)
    The problem does NOT occur on macOS, as far as my testing goes.

  3. Problem:
    Rocket seems to be dropping connections on high cpu usage / on some other variables we haven't been able to identify, which causes our reqwest based client to panic in a way that shouldn't occur.

Expected: Rocket to close out connections correctly (when setting keep_alive to 0, this problem doesn't occur.

  1. We were using our rocket based backend and a reqwest based client and our tests seemed to sometimes, almost randomly fail. Basically the client is just a struct / impl pair that is initialized once to get a client instance, and it uses a reqwest client instance to make requests.

  2. Current guess is that on low available cpu resources / high cpu usage (or some other variable that happens more often on travis / my VM than in more resourceful contexts) rocket drops some connections without properly informing the client that they've been closed.

General Comments

I didn't really write a minimal repro case yet because I'd first like somebodys input who's well versed in Rocket on if this is something that can / should happen, ever. I couldn't find any issues on the topic but if this is an already known / intended behaviour I'd like to know about it.

I'll look into writing the minimal repro case but would make it much easier if I knew what details here are the unkowns from Rocket's perspective.

@jebrosen
Copy link
Collaborator

jebrosen commented Mar 7, 2020

See my comment at #1023 (comment) - rocket 0.5 will be async and based on a maintained version of hyper, which together solve a lot of issues like these.

Setting keep-alive to 0 and/or running a rocket server behind a reverse proxy such as nginx are currently the easiest workarounds I know of.

@jebrosen jebrosen added the duplicate This issue or pull request already exists label Mar 7, 2020
xylix added a commit to xylix/aw-server-rust that referenced this issue Mar 8, 2020
xylix added a commit to xylix/aw-server-rust that referenced this issue Mar 8, 2020
johan-bjareholt pushed a commit to ActivityWatch/aw-server-rust that referenced this issue Mar 9, 2020
@SergioBenitez
Copy link
Member

@xylix I've seen this as well, and short of switching the underlying HTTP library, there's nothing we can do as the issue lies with hyper. Setting keep_alive to 0 is the work-around.

As the issue should be resolved entirely in 0.5, and should already be resolved in the async branch, I'm closing this out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
duplicate This issue or pull request already exists
Projects
None yet
Development

No branches or pull requests

3 participants