-
Notifications
You must be signed in to change notification settings - Fork 153
Add a stress test for uninit representations #507
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Addressed both of the review comments. |
Will merge with CI passing. |
@HeroicKatora how long does that test take to run on your system (and how long does ctfe-stress-2 take)? |
Hm, just copy/pasting the source code leads to a few compilation failures; looks like CI isn't catching it but https://travis-ci.com/rust-lang-nursery/rustc-perf/jobs/245574193#L1228 is a sample log |
Turns out I am bad a copy&paste, the tests were mostly built from perf files I had flying around for my own testing and I didn't properly validated. Thanks CI. After the import fix, current exection was ~4 seconds, so I've reduce the shift/size by |
Yes, the specific execution time is not really relevant (due to differences in CPU speed etc); you probably want to be somewhere around the time that ctfe-stress-2 takes to execute locally for you. |
|
Wait what? No. This is like one of the tests in ctfe-stress-2, which are all geared to around 2s on my system (or rather they were when I originally submitted it). That said, are these 4s with or without your PR? If that PR is going to cut the time down in half it may be worth factoring that in already. |
Ah, right, I forget that we had that many tests in there. In that case, yes; we will want just 1/10 or so. We can use timings from the PR rather than nightly, though. |
This perf contains more than one case also, some that will be directly affected by the PR. Some other cases are not yet fully affected and included to ensure that future improvements don't need to add their own stress test crates, I'd say the ratio of directly affected code is about 50/50. |
With the PR, the timing difference between ctfe-stress-2 and ctfe-stress-uninit is locally a factor of 7.2 (13.6s vs. 1.9s). |
Can we just add this new code to |
Which of the two options should I go with, then? Any differing opinions? |
I think in this case -3 is probably the best option as it sidesteps most of the concerns and is presumably fairly easily. |
Sounds good to me. |
Sure, easy enough. |
collector/benchmarks/README.md
Outdated
@@ -68,6 +68,8 @@ programs. | |||
caused [poor performance](https://github.com/rust-lang/rust/issues/32278) in | |||
the past. | |||
- **ctfe-stress-2**: A stress test for compile-time function evaluation. | |||
- **ctfe-stress-uninit**: A stress test for representation of values with | |||
uninitialized bytes in compile-time function evaluation. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be updated.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove it or merge the two comments?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is no uninit benchmark anymore, I would expect a single entry with -3.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Probably just dropping the uninit bit; that can be moved to a comment in the file.
Test ctfe that never initializes a large portion of its value representations. This might be optimized independent of other changes to ctfe. The potential usefulness of having these benchmarks to evaluate such changes was discussed in rust-lang/rust#62655
cc: @oli-obk @RalfJung @Mark-Simulacrum