8359646: C1 crash in AOTCodeAddressTable::add_C_string #25841

vnkozlov · 2025-06-17T02:34:24Z

It is concurrency issue. Call to AOTCodeAddressTable::add_C_string() happened after checks that AOT code cache is still opened. But, because there is no synchronization, other thread (VM) closed/delete AOT code cache (after dumping) before code in add_C_string() accessed it.

Added missed AOTCodeCStrings_lock in places where we modify, store and delete AOT strings table. Moved MutexLocker from AOTCodeAddressTable::add_C_string() to its caller and do additional check after it.

I also noticed that we missed similar check after Compile_lock when we are storing AOT code.

Tested hs-tier1-6,hs-tier10-rt,stress,xcomp

Progress

Change must be properly reviewed (1 review required, with at least 1 Reviewer)
Change must not contain extraneous whitespace
Commit message must refer to an issue

Issue

JDK-8359646: C1 crash in AOTCodeAddressTable::add_C_string (Bug - P3)(⚠️ The fixVersion in this issue is [25] but the fixVersion in .jcheck/conf is 26, a new backport will be created when this pr is integrated.)

Reviewers

Andrew Dinn (@adinn - Reviewer)
Ioi Lam (@iklam - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/25841/head:pull/25841
$ git checkout pull/25841

Update a local copy of the PR:
$ git checkout pull/25841
$ git pull https://git.openjdk.org/jdk.git pull/25841/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 25841

View PR using the GUI difftool:
$ git pr show -t 25841

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/25841.diff

Using Webrev

Link to Webrev Comment

bridgekeeper · 2025-06-17T02:34:51Z

👋 Welcome back kvn! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2025-06-17T02:35:06Z

@vnkozlov This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8359646: C1 crash in AOTCodeAddressTable::add_C_string

Reviewed-by: adinn, iklam

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 11 new commits pushed to the master branch:

53a83d1: 8357799: Improve instructions for JFileChooser/HTMLFileName.java
21b72de: 8357862: Java argument file is parsed unexpectedly with trailing comment
51877f5: 8359207: Remove runtime/signal/TestSigusr2.java since it is always skipped
... and 8 more: https://git.openjdk.org/jdk/compare/e57a214e2a1059109dd028369d518298cfa5d5b4...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

openjdk · 2025-06-17T02:35:41Z

@vnkozlov The following label will be automatically applied to this pull request:

hotspot-compiler

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

vnkozlov · 2025-06-17T02:38:13Z

@adinn and @ashu-mehra please look.

Very rare case because very narrow window for concurrency.

vnkozlov · 2025-06-17T02:38:37Z

This happens during assembly phase.

mlbridge · 2025-06-17T02:41:44Z

Webrevs

00: Full (d283ef27)

adinn · 2025-06-17T08:10:43Z

src/hotspot/share/code/aotCodeCache.cpp

@@ -819,6 +820,9 @@ bool AOTCodeCache::store_code_blob(CodeBlob& blob, AOTCodeEntry::Kind entry_kind
  // we need to take a lock to prevent race between compiler threads generating AOT code
  // and the main thread generating adapter
  MutexLocker ml(Compile_lock);
+  if (!is_on()) {


Just for the record:

I can see why this is needed to stop compiler threads dereferencing a null cache pointer or null table pointer when some other thread might concurrently be closing the cache.

That led me to wonder why we don't need to further synchronize concurrent execution of non-compiler threads. I convinced myself that whenever a non-compiler thread calls AOTCodeCache::close() the only other running threads that might try to access the AOT cache are compiler threads -- calls to the close() method are from MetaspaceShared::preload_and_dump_impl (metaspaceShared.cpp), before_exit (java.cpp) and Threads::create_vm (threads.cpp). (Well, modulo a rogue jcmd, perhaps?).

Yes. Ans with jcmd, as @iklam pointed in an other PR, "that would create far worse problems than the bug that we are trying to fix here" (https://git.openjdk.org/jdk/pull/25816#issuecomment-2975238342)

adinn

Looks good

ashu-mehra · 2025-06-17T14:16:21Z

Unless I missed something, I think there is still synchronization problem between the thread adding a string to the AOTCodeCache and the thread closing the AOTCodeCache. See the following pattern of execution:

t0: Thread T1 calls add_C_string(), checks for is_on_for_dump which returns true
t1: Thread T2 calls close() and completes the call to finish_write()
t2: T1 acquires AOTCodeCStrings_lock and checks that _table is not null which returns true
t3: T2 clears the table and sets it to null
t4: T1 tries to call table->add_C_string

I think at the time of shutting down the AOTCodeCache, the thread should hold both Compile_lock and AOTCodeCStrings_lock.

vnkozlov · 2025-06-17T15:10:29Z

t3: T2 clears the table and sets it to null

It can't because T1 holds the lock. See that I added the lock before delete _table.

vnkozlov · 2025-06-17T15:10:49Z

Thank you @adinn and @ashu-mehra for review.

iklam

LGTM

vnkozlov · 2025-06-17T15:37:34Z

Thank you, @iklam

vnkozlov · 2025-06-17T15:53:44Z

/integrate

openjdk · 2025-06-17T15:55:19Z

Going to push as commit 9607021.
Since your change was applied there have been 11 commits pushed to the master branch:

53a83d1: 8357799: Improve instructions for JFileChooser/HTMLFileName.java
21b72de: 8357862: Java argument file is parsed unexpectedly with trailing comment
51877f5: 8359207: Remove runtime/signal/TestSigusr2.java since it is always skipped
... and 8 more: https://git.openjdk.org/jdk/compare/e57a214e2a1059109dd028369d518298cfa5d5b4...master

Your commit was automatically rebased without conflicts.

openjdk · 2025-06-17T15:55:33Z

@vnkozlov Pushed as commit 9607021.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

ashu-mehra · 2025-06-17T17:36:37Z

It can't because T1 holds the lock. See that I added the lock before delete _table.

yup, I missed that lock. All good then.

8359646: C1 crash in AOTCodeAddressTable::add_C_string

d283ef2

openjdk bot added the rfr Pull request is ready for review label Jun 17, 2025

openjdk bot added the hotspot-compiler [email protected] label Jun 17, 2025

adinn reviewed Jun 17, 2025

View reviewed changes

adinn approved these changes Jun 17, 2025

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Jun 17, 2025

iklam approved these changes Jun 17, 2025

View reviewed changes

openjdk bot added the integrated Pull request has been integrated label Jun 17, 2025

openjdk bot closed this Jun 17, 2025

openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Jun 17, 2025

vnkozlov deleted the 8359646 branch June 17, 2025 15:59

8359646: C1 crash in AOTCodeAddressTable::add_C_string #25841

8359646: C1 crash in AOTCodeAddressTable::add_C_string #25841

Conversation

vnkozlov commented Jun 17, 2025 • edited by openjdk bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Progress

Issue

Reviewers

Reviewing

Uh oh!

bridgekeeper bot commented Jun 17, 2025

Uh oh!

openjdk bot commented Jun 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openjdk bot commented Jun 17, 2025

Uh oh!

vnkozlov commented Jun 17, 2025

Uh oh!

vnkozlov commented Jun 17, 2025

Uh oh!

mlbridge bot commented Jun 17, 2025

Webrevs

Uh oh!

adinn Jun 17, 2025

Choose a reason for hiding this comment

Uh oh!

vnkozlov Jun 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adinn left a comment

Choose a reason for hiding this comment

Uh oh!

ashu-mehra commented Jun 17, 2025

Uh oh!

vnkozlov commented Jun 17, 2025

Uh oh!

vnkozlov commented Jun 17, 2025

Uh oh!

iklam left a comment

Choose a reason for hiding this comment

Uh oh!

vnkozlov commented Jun 17, 2025

Uh oh!

vnkozlov commented Jun 17, 2025

Uh oh!

openjdk bot commented Jun 17, 2025

Uh oh!

openjdk bot commented Jun 17, 2025

Uh oh!

ashu-mehra commented Jun 17, 2025

Uh oh!

Uh oh!

vnkozlov commented Jun 17, 2025 •

edited by openjdk bot

Loading

openjdk bot commented Jun 17, 2025 •

edited

Loading

vnkozlov Jun 17, 2025 •

edited

Loading