8356865: C2: Unreasonable values for debug flag FastAllocateSizeLimit can lead to left-shift-overflow, which is UB

[benoitmaillard]

This PR adds a range constraint for the -XX:FastAllocateSizeLimit debug flag. This prevents undefined behavior caused by left-shift overflow of the flag value in GraphKit::new_array.

Testing

• GitHub Actions
• tier1-3, plus some internal testing
• Manual testing with values known to previously cause undefined behavior

Thanks!

---

Progress

• Change must be properly reviewed (1 review required, with at least 1 Reviewer)
• Change must not contain extraneous whitespace
• Commit message must refer to an issue

Issue

• JDK-8356865: C2: Unreasonable values for debug flag FastAllocateSizeLimit can lead to left-shift-overflow, which is UB (Bug - P4)

Reviewers

• Manuel Hässig (@mhaessig - Author) 🔄 Re-review required (review applies to 486ddcb6)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/25834/head:pull/25834
$ git checkout pull/25834

Update a local copy of the PR:
$ git checkout pull/25834
$ git pull https://git.openjdk.org/jdk.git pull/25834/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 25834

View PR using the GUI difftool:
$ git pr show -t 25834

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/25834.diff

Using Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back bmaillard! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

❗ This change is not yet ready to be integrated.
See the Progress checklist in the description for automated requirements.

[openjdk]

@benoitmaillard The following label will be automatically applied to this pull request:

• hotspot

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

[mlbridge]

Webrevs

• 01: Full - Incremental (c8904a29)
• 00: Full (486ddcb6)

[benoitmaillard]

/label add hotspot-compiler

[openjdk]

@benoitmaillard
The hotspot-compiler label was successfully added.

[mhaessig]

Thank you for working on this, @benoitmaillard. This looks good to me.

[eastig]

Prior C++14 left shit producing a negative value is undefined behavior: https://www.open-std.org/jtc1/sc22/wg14/www/docs/n2161.pdf

Do we compile c++ source specifying the C++ standard?

[dean-long]

Yes we use -std=c++14, but creating a negative value in this way still feels like a kind of overflow to me.

[benoitmaillard]

Thanks for the comments!

I added the assert because the issue in the JBS mentioned a specific case where we ended up with negative values.

Should I leave it like this, or rather convert it to a more specific check (ie. making sure that the LogBytesPerLong - log2_esize most significant bits are not used before shifting)?

[eastig]

IMO your assert is obfuscating the overflow problem.
I think the assert should be before doing the shift.
It can be like:

assert((fast_size_limit == 0) || (count_leading_zeros(fast_size_limit) > (LogBytesPerLong - log2_esize), "fast_size_limit (%d) overflow when shifted left by %d", fast_size_limit, (LogBytesPerLong - log2_esize));