-
Notifications
You must be signed in to change notification settings - Fork 105
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Basic performance benchmark using criterion for pub sub and req rep #81
Conversation
@TheButlah IMO PUB/SUB (especially 10x of them) could be a bit hard to start with. I would suggest to start with some kind of really simple scenario like 1 REQ - 1 REP. This is much easier to setup and debug. Currently I don't have any idea why it blocks. I can take a closer look on it on a weekend. Currently I have to write code for money not for fun :( |
OK, I've added a benchmark for the req rep pattern. As you predicted, it was a lot easier than Pub Sub, and unlike the other one, it actually works. I've left the pub sub benchmark code in this PR, but have commented it out from the Cargo.toml so that it will not be run on A couple of observations I can make: Even with a sample size of 128 and a benchmark time of 32 seconds with 5 secs warmup time (that gives 512 non-warmup iterations), criteron frequently reports statistically significant changes in performance, when no such changes should be present. I can only assume that this is because of the fluctuating performance of the underlyling tcp sockets on the host computer. We may want to mock these out from tokio in the future, to avoid this source of nondeterminism |
@TheButlah I've tested this branch on my PC. I also see fluctuations but it seems that they are mostly related to my workload and Youtube playing in background.. On idle PC I have 3% or less difference between runs. Looks like a good start. We can run the same tests for your codec branch and see is there any difference |
Resolves #80
This PR adds a basic performance benchmark using criterion.rs.
Currently, it hangs on the first
s.recv()
in therecv_msgs()
function. I can't figure out why it is hanging here. Any ideas @Alexei-Kornienko?